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Abstract 

A linear-quadratic (LQ, for short) optimal control problem is considered for mean-field stochastic 
differential equations with constant coefficients in an infinite horizon. The stabilizability of the control 
system is studied followed by the discussion of the well-posedness of the LQ problem. The optimal control 
can be expressed as a linear state feedback involving the state and its mean, through the solutions of two 
algebraic Riccati equations. The solvability of such kind of Riccati equations is investigated by means of 
semi-definite programming method. 
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1 Introduction. 

Let P, F) be a complete filtered probability space, on which a one-dimensional standard Brownian 

motion W(-) is defined with F = {J-t} t >o being its natural filtration augmented by all the P-null sets. 
Consider the following controlled linear stochastic differential equation (SDE, for short) in R": 

dX(t) = $AX(t) + AE[X(t)] + Bu(t) + EE[u(t)]jdt 

+{cX(t) + CE[X(t)] + Du(t) + DE[u(t)]}dW(t), t > 0, ( L1 ) 
,X(0) = x, 

where A, A, C,C € R" xn and B,B,D,D e R" xm arc given (deterministic) matrices. In the above, X(-), 
valued in R n , is called the state process, and u(-), valued in R m , is called a control process. 

Different from classical controlled linear SDEs, the terms E[X(-)] and E[it(-)] appear in the equation. 
We call (1.1) a controlled mean-field (forward) SDE (MF-FSDE, for short). Historically, a special case of 
MF-FSDE, called McKean-Vlasov SDE, was suggested by Kac [23] in 1956 as a stochastic toy model for the 
Vlasov type kinetic equation of plasma and the rigorous study of which was initiated by McKean [28] in 1966. 
Since then, such kind of equations were studied by many authors, see, for examples, Dawson [17], Dawson- 
Gartner [18], Gartner [20], Scheutzow [32], Graham [21], Chan [14], Chiang [15], Ahmed-Ding [2], and the 
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references cited therein. For the relevant works of recent years, see, for examples, Veretennikov [34], Huang- 
Malhame-Caines [22], Buckdahn-Djchiche-Li-Peng [12], Buckdahn-Li-Peng [13], Borkar-Kumar [9], Crisan- 
Xiong [16], Kotelenez-Kurtz [25], and so on. Control problems of McKcan-Vlasov equation or MF-FSDEs 
were investigated by Ahmed-Ding [3], Ahmed [4], Buckdahn-Djehiche-Li [11]), Park-Balasubramaniam- 
Kang [30], Andersson-Djehiche [6], Meycr-Brandis-Okscndal-Zhou [29], and so on. In Yong [35], a linear- 
quadratic (LQ, for short) problem was introduced and investigated for MF-FSDEs in finite horizons. Some 
interesting motivation was given in [35] for the control problem with E[A(-)] and E[u(-)] being included in 
the cost functional. This paper can be regarded as a continuation of [35], for LQ problem of MF-FSDEs in 
an infinite horizon. 



We introduce the following: 



U[0,T] = ju : [0,oo) x SI 
Z4> c [0,oo) = |J U[0,T], 



u(-) is F-adapted, E^ \u(s)\ 2 ds < oo}, VT > 0, 



T>0 



W[0,oo) = ju(-) eW ioc [0,oo) | E / \u(s)\ 2 ds < oo} 



Any u(-) £ Ui oc [Q, oo) is called a control process and any u(-) G U[0, oo) is called a feasible control process. 
Likewise, we define 



X[0,T] = | A : [0,oo) x fi -> 1 
X loc [0,oo) = |J X[0,T], 

T>0 

X[0,oo) = [X(-) 6 X loc [0,oo) 



X(-) is F-adapted, t \-t X(t,ui) is continuous, 

E[ sup \X{t)\ 2 ] < oo 
te[o,r] 



T > 0, 



\X{t)\ 2 dt < oo}. 

Any element in <Y; OC [0, oo) is called a state process. It is not hard to see that 



W[0,oo) C W /oc [0,oo), 
A"[0,oo) C Xi oc [0,oo), 



W[0,oo) ^W /oc [0,oo), 
A"[0,oo) ^ A" /oc [0,oo). 



By a standard argument using contraction mapping theorem, one can show that for any (x,u{-)) £ R™ x 
Ui oc [Q, oo), (1.1) admits a unique solution X(-) = X(-;x, u(-)) € A/ oc [0,oo). Next, we let Q,Q € S n and 
R,RG S m , where S k is the set of all symmetric matrices of order (k x k), and introduce the following cost 
functional: 

J(x;u(-))=E / I (QA( s ),A( s )) + (QE[A(s)],E[A(s)]) 

7o L (1.2) 

+ ( Ru(s), u(s) ) + ( M[u(s)],E[u(s)] ) }ds, 

where X(-) = X(- ;x,u(-)) on the right hand side of the above. Note that in general, for (x,u(-)) 6 M. n x 
U[0, oo), the solution X(-) = X(- ; a;, u(-)) of (1.1) might just be in A; oc [0, oo) and the above cost functional 
J(x;u(-)) might not be defined. Therefore, we introduce the following: 



a d [0,oo) = S W[0,oo) I J(x;u(-)) is defined, Vx £ R™}. 

Any element it(-) € lA a d[Q, oo) is called an admissible control process and the corresponding X(-) = X{- ; x, u(-)) 
is called an admissible state process. We see that the structure of U ac i[0,oo) is very complicated, since it 
involves not only the state equation, but also the cost functional. Some better description of U a d[0,oo) will 
be given a little later, under proper conditions. Our optimal control problem can be stated as follows: 



Problem (MF-LQ). For given x G R n , find a «*(•) G U ad 



[0, oo) such that 



J(x;u„(-)) 



inf 

«(-)e"ad[o,oo) 



J(a:;«(.))=V(a:). 



Any it*(-) G W a d[0,oo) satisfying the above is called an optimal control process and the corresponding 
state process -X*(-) = X(- ; x, «*(•)) is called an optimal state process; the pair (_X*(-), «*(■)) is called an 
optimal pair. The function V(-) is called the value function of Problem (MF-LQ). 

It is not hard to see that in order Problem (MF-LQ) to make sense, we need U a d[0, oo) to be nonempty, 
at least. To achieve this, we will carefully discuss various stabilizability (for which both the state equation 
and the cost functional are involved) of the controlled MF-FSDE (1.1), which are interestingly different 
from the classic ones, due the the appearance of the terms E[X(-)] and E[u(-)]. Once the set U a d[0,oo) of 
admissible controls is nonempty, under some standard assumptions, we are able to show that the optimal 
control uniquely exists. Then inspired by the results of [35], we obtain a system of algebraic Riccati equations 
(AREs, for short), whose solutions will lead us to the state feedback representation of the optimal control. 
The existence of the solutions to the derived ARE system is established under some reasonable conditions. 
Our results recovers relevant ones for the classic linear-quadratic optimal controls of SDEs. 

The rest of the paper is organized as follows. Section 2 collects some preliminary results concerning the 
state equation. In Sections 3 and 4, the stability and the stabilizability of the state equation are discussed. In 
Section 5, Problem (MF-LQ) is solved by means of AREs. In Section 6, the solvability of AREs is discussed 
by linear matrix inequalities (LMIs, for short). A couple of numerical examples are presented in Section 7. 
Finally, some supporting results for Section 6 are listed in the Appendix. 

2 Preliminary Results 

In this section, we present some preliminary results. First of all, let us consider the following result, whose 
proof follows a standard argument using contraction mapping theorem, together with Ito's formula. 

Proposition 2.1 For any x G R" and u(-) G W[0, oo), there exists a unique X(-) = X(-;x, u(-)) solving 
(1.1). Moreover, 



where Lt > is a constant depending on T, and independent of (cc, «(■)). 

For later purposes, we make some calculations. Let X(-) = X(- ; x, u(-)) be the solution of (1.1). For any 
deterministic diffcrentiable function P(-) valued in S n , by Ito's formula, we have 
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Thus, 

E ( P(t)X(t),X(t) ) = ( P(0)x, x)+eJ { ( [P(s) + P(s)A + A T P(s) + C T P(s)C]X(s), X(s) ) 
+2 ( u(s), {B T P{s) + D T P{s)C] X(s) ) + ( D T P{s)Du(s),u(s) ) 

+ ( [P(s)A + A T P(s) + C T P(s)C + C T P(s)C + C T P(s)C]E[X(s)},E[X{s)} } C 2 ' 1 ) 
+2 (E[u(s)), [B T P(s) + D T P{s)C + D T P(s)C + D T ' P(s)C]E[X(s)] } 
+ ( [D T P(s)D + D T P(s)D + D T P(s)D] E[u(s)} , E[u(s)] }}ds. 

Also, 



(2.2) 



{ P(t)E[X(t)],E[X(t)] ) = ( P(0)x, x)+J {( [P(s) + P(s)(A+A) + (A+A) T P( s )]E[X( s )} 1 E[X( s )} ) 

+2 (E[u(s)], (B + B) T P(s)E[X(s)} )}ds. 
Combining (2.1) and (2.2), we obtain 

E(P(t){X(t) -E[X(t)]},X(t) -E[X(t)}) = E(P{t)X(t),X(t)) - (P(t)E[X(t)],E[X(t)} ) 

= eJ {( [P{s) + P(s)A+ A T P( S ) + C T P(s)C]X(s),X(s)) +2 (u(s), [B T P(s) + D T P(s)C]X(s)} 

+ (D t P(s)Du(s),u(s))+([P(s)A+A t P(s) + C t P(s)C+C t P(s)C+C t P(s)C]E[X^ 
+2 (E[u(s)], [B T P(s) + D T P(s)C+D T P(s)C+D T P(s)C]E[X(s)} ) 
+ ([D t P(s)D + D t P(s)D + D t P(s)D]E[u(s)},E[u(s)}) 

- ( [P(s) + P(s)(A + A) + (A + A) T P(s)]E[X(s)},E[X(s)} ) -2 (E[u(s)}, (B + B) T P(s)E[X(s)} ) }ds 

= eJ\ ( [P( s )+P( s )A+A T P( s )+C T P( s )C]{X(s) -E[X(s)]},X(s) -E[X(s)}) 

+2 (u(s)-E[u(s)], [B T P(s)+D T P(s)C] {X (s) -E[X (s)}}) 
+ (D T P(s)D{u{s)-E[u(s)}},u(s)-E[X(s)} > 

+ (P(s){(C+C)E[X(s)} + (D + D)E[u(s)]}, (C+C)E[X(s)} + (D + D)E[u(s)} ) }ds. 
In the case that P(t) = P e <S™, we have 



E ( PX(t), X(t) ) = ( Px, x ) +EJ { ((PA + A T P + C T PC)X(s), X(s) ) 
+2(u(s), (B T P+D T PC)X(s) } + ( D T PDu(s),u(s) ) 

+ ((PA+A T P+C T PC+C T PC+C T PC)E[X(s)},E[X(s)} ) ( 2 ' 3 ) 
+2 (E[it(s)], (B T P + D T PC + D T PC + D T PC)E[X(s)] ) 
+ ((D T PD + D T PD + D T PD)E[u(s)],E[u(s)} ) }<is, 



\PE[X(t)],E[X(t)}) = (Px,x) +j { ( [P(A+A) + (A+A) T P]E[X(s)],E[X(s)}) 
+2 (E[u(s)], (B + B) T PE[X(s)} )|ds 
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and 

E(P{X(t) -E[X(t)]},X(t) -E[X(t)}) 

= eJ* { ((PA + A T P + C T PC){X( S ) - E[X(s)]},X(s) E[X(s)] ) 

+2(u(s)-E[u(s)},(B T P+D T PC){X(s)-E[X(s)}} ) + ( D T PD{u(s) -E[«(s)]}, u(s) -E[X(s)] ) 
+ (P{(C+C)E[X(s)] + (D + D)E[u(s)}},(C+C)E[X(s)} + (D + D)E[u(s)} ) }ds. 

The above will be useful later. 

Now, let us look at the cost functional. We observe that the cost functional J(x;u(-)) defined by (1.2) 
can also be written as 

J(x;u(-)) =eJ { (Q{X(t) -E[X(t)]},X(t) -E[X(t)}} + ((Q + Q)E[X(t)],E[X(t)]) 

+ (R{u(t) - E[u{t)]},u(t) - E[u(t)] ) + ((R + R)E[u(t)],E[u(t)] ) jrfi. 

In what follows, when the dimension of a matrix, say, Q is clear from the context, we write Q > for Q E S n 
being positive semi-definite and write Q > for Q € S n being positive definite. We now introduce the 
following assumption concerning the weighting matrices Q,Q, R, R in the cost functional. 

(J) The matrices Q,Q £ S n and R,R(E S m satisfy the following: 

Q,Q + Q>0, R,R + R>0. 

Note that in (J), we do not have direct assumption on Q and R, they do not have to be positive (semi-) 
definite, and actually, they could even be negative definite. Under (J), we see that u(-) E U a d[0,oo) if and 
only if for any x E R", the corresponding state process X(-) = X(- ; x, u(-)) satisfies 

E (\Q^{X(t)-E[X(t)}}\ 2 + \(Q + Q)^E[X(t)]\ 2 yt<oo. (2.5) 

Since Q and/or (Q+Q) might be degenerate, when u(-) E U a d[0, co), we might not have X(-) = X(-, ; x, u(-)) E 
X[0, co). The following is a little stronger assumption than (J). 

(J)' The matrices Q, Q E S n and R, R e S m satisfy the following: 

Q,Q + Q>0, R,R + R>0. 

Clearly, if (J)' holds, then u(-) E U a d[0, co) if and only if for any x E M", X(- ; x, u(-)) E ^[0, oo). 

3 Stability 

Now, let us return to state equation (1.1). We know that cost functional J(x;u(-)) is well-defined on 
R" x W o d[0, co), and unlike U[0, co), the structure of U a d[0, oo) seems to be complicated since it involves the 
state equation and the cost functional. Further, the following example shows that U a d[0,oo) could even be 
empty, which leads to that Problem (MF-LQ) is meaningless. 

Example 3.1 Consider one-dimensional controlled system: 

dX(t) = X(t)dt + {E[X(t)} + u(t)}dW(t), t > 0, 
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t > 0, 



with cost functional 

/■oo 

J(x;u(-)) =E / \X(t)\ 2 dt. 
Jo 

Clearly, 

dE[X(t)} =E[X(t)]dt, t>0, 

which implies 

E[X{t)]=xe t , t>0. 

Then 

dX(t) = X(t)dt + [xe* + u(t)]dW(t), t > 0. 

Hence, 

X{t) = xe 1 + e'- s [xe s + u(s)]dW(s) = e*|x + 
and as long as x ^ or u(-) ^ 0, 

J{x;u{-))=Ej \X(t)\ 2 dt = J e 2t {x 2 +j [x + e~ s u( s )] dsjdt = oo. 

Therefore, in this case, Z// a£ i[0, exo) = 0. Consequently, the corresponding Problem (MF-LQ) is not meaningful. 

From the above, we see that before investigating Problem (MF-LQ), we should find conditions for the 
system and the cost functional so that the set U a d[0, oo) is at least non-empty and hopefully it admits an 
accessible characterization. To this end, let us first look at the following uncontrolled linear MF-FSDE 
(which amount to saying that taking u(-) — or letting B = B = D = D = 0): 

dX (t) = ^AX{t) +M[X(t)]jdt+ \cX{t) + CE[X(t)]jdW{t), t>0, 
X(0) = x, 

where A, A,C,C £ K nx ™ are given matrices. The above uncontrolled system is briefly denoted by [A, A, C, C]. 
For simplicity, we also denote [A, C] = [A,Q, C, 0] (the linear SDE without mean- fields), and A = [A,0] = 
[A, 0,0,0] (the linear ordinary differential equation, ODE, for short). Let us now introduce the following 
definition. 

Definition 3.2 (i) System [A,A,C,C] is said to be L 2 - exponentially stable if for any x £ R™, the solution 
X(-) = X(- ; x) £ <Y; OC [0, oo) of (3.1) satisfies the following: 

lim e xt E\X(t)\ 2 = 0, 

t— >oo 

for some A > 0. 

(ii) System [A, A, C, C] is said to be L 2 -globally integrable if for any x £ R", the solution X(-) = X(- ; x) £ 
A/ oc [0, oo) of (3.1) is in X[0, oo), namely, 



/•oo 

/ E\X(t)\ 2 dt < oo. 



(iii) System [A,A,C,C] is said to be L 2 - asymptotically stable if for any x £ R™, the solution X(-) = 
X(- ; x) £ Xioc\fi, oo) of (3.1) satisfies the following: 

lim E\X{t)\ 2 = 0. (3.2) 

(iv) Let (J) hold. System [A, A, C, C] is said to be L 2 ^ q- globally integrable if for any x £ R™, the solution 
X(-) =X(-;x) £ ^, oc [0,oo) of (3.1) satisfies (2.5). 
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It is clear that the above (i)-(iii) can be defined for linear SDE system [A, C] = [A, 0, C, 0], and linear 
ODE system A = [A, 0]. By a standard result, we know that the above (i)-(iii) are equivalent for linear 
ODEs. For general linear MF-SDEs, we have the following result. 

Proposition 3.3 Among the following statements: 

(i) System [A, A, C, 0] is L 2 -exponentially stable; 

(ii) System [A, A,C, C] is L 2 -globally integrablc; 

(iii) System [A, A, C, C] is L 2 -asymptotically stable; 

(iv) Let (J) hold. System [A, A, C, C] is Lg Q-globally integrablc. 
The following implications hold: 

(i) => (ii) => (iii); 

when (J) holds, (ii) => (iv); when (J)' holds, (iv) => (ii). 

Proof. The implications (i) =4> (ii) and (ii) =>• (iv) (under (J)) are clear. It is also clear that under (J)', 
(iv) => (ii). We now prove (ii) => (iii). Note that 



E\X(t)\ 2 = \x\ 2 +E j (2(X(s),AX{s) + AE[X(s)})+\CX{s) + CE[X{s)}\ 2 y s 

L E\X(s)\ 2 ds < \x\ 2 + LE / \X(s)\ 2 ds. 
Jo Jo 



<\x\ 2 



Hereafter L > stands for a generic constant which could be different from line to line. Thus, E|X(£)| 2 is 
bounded uniformly in t 6 [0, oo). Consequently, for any < r < t < oo, 



E\X{t)\ 2 -E\X(t)\ 2 <E j (2\(X(s),AX(s) + AE[X(s)})\ + \CX(s) + CE[X{s)}\ 2 ^ds 
<L(t-r). 

Hence, t M> E|X(t)| 2 is uniformly continuous on [0,oo), which, together with the integrability of E|X(-)| 2 
over [0, oo), leads to (3.2). □ 

Let us make the following remarks. 
L 2 -global integrability of the system in general 



• When (J) holds but (J)' does not hold, the Lg g-global integrability of the system does not imply the 



• It is not clear if (iii) implies (ii), although these two are equivalent for ODE case. 

• The notion that is the most relevant to our Problem (MF-LQ) is the L 2 ^-global integrability. 

Our next goal is to explore when (ii) implies (i). To this end, we first look the case A = C = 0. In this 
case, our system becomes system [A, C]: 

j dX{t) = AX{t)dt + CX{t)dW{t), t>0, 
{ X(0) = x. 

For such a system, instead of Lq g-global integrability, we may introduce the following. 
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Definition 3.4 Let Q > 0. System [A, C] is said to be Lg-globally integrable if for any x £ R™, the solution 
X(-) ee X(-;x) of (3.3) satisfies 

p OO 

E / (QX(t),X(t))dt < oo. 
Ja 

In the case that Q > 0, the Lg-global integrability is simply called the L 2 -global integrability which is 
equivalent to X(- ; x) £ X[0, oo) for all ifR™. 

We have the following result concerning the Lg-global integrability of [A, C]. 

Proposition 3.5 Let Q > 0. Then the following arc equivalent: 

(i) System [A, C] is Lg-globally integrable; 

(ii) The following Lyapunov equation admits a solution P > 0: 

PA + A T P + C T PC + Q = 0; (3.4) 

(iii) The Lyapunov equation (3.4) admits a solution P £ S n and 

Ihn" E\X(t;x)\ 2 < oo, Vi £ K", 

t— loo 

which is the case, in particular, if [A, C] is L 2 -asymptotically stable. 

In the above case, the solution P of the above equation admits the following representation: 



I' / F(ty QF(t)dt, (3.5) 
Jo 

where F(-) is the solution to the following: 

j dF(t) = AF(t)dt + CF(t)dW(t), t £ [0, oo), 



i 



F(0) = I. 



The above result should be standard. However, since the idea contained in the proof will be useful below, 
for readers's convenience, we present a proof here. 

Proof, (i) (ii). Suppose system [A, C] is Lg-globally integrable. We want to show that Lyapunov 
equation (3.4) admits a solution P > 0. To this end, let us consider the following linear ODE: 



f -0(i) + Q{t)A + A T Q{t) + C T Q{t)C + Q = 0, t £ [0, oo), 

\e(o) = o, 

which has a unique solution &(■) defined on [0, oo). For any fixed r > 0, we define 

6 T (s) =6(r-s), se[0,r]. 
Then T (-) is the solution to the following: 

6 T (s) + Q r (s)A + A T Q r (s) + C T Q T {s)C + Q = 0, s £ [0, r], 
9 r (r) =0. 



(3.6) 
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For any x G W 1 , letX(-) = X(-;x) be the solution of (3.3). Applying Ito's formula to s h-> ( @ T (s)X(s), X(s) ), 
one has 

- ( Q(t)x, x} = -( e T (Q)x, x)=e\( Q t (t)X(t), X(t) ) - ( 9 r (0)X(0), X(0) ) 

= e f ({e T ( s ) + e T (s)A + A T e T (s) + c T e T (s)c}x( s ),x(s))ds 

Jo 

. / (QX(s),X(s))ds = -E / (F{s) T QF{s)x,x)ds. 
Jo Jo 

Thus, the solution 0(-) of (3.6) admits the following representation: 

9(t) = E [ F(s) T QF{s)ds, r > 0. 
Jo 

From the above, since Q > 0, we sec that r M> 6(r) is non-decreasing and by the Lg-global integrability of 
[A, C], one has the following limit: 

/•OO 

lim 9(t) = E / F(s) T QF(s)ds = P. 

We claim that such a P > must be a solution to the Lyapunov equation (3.4). In fact, from (3.6), one has 

et+i ,t+i ^t+i 

e(t)-e(t + i) + [y e(s)ds)A + A T (J e( s )dsj+c T ^J e( s )dsjc + Q = o. 

Letting t — )• oo, we see that (3.4) is satisfied by P. 

(ii) => (i) Suppose there exists a P > satisfying (3.4). Then 



E ( PX(t), X{t) ) - ( Px, x ) = E [ {(PA + A T P + C T PC)X(s), X(s) } ds 

Jo 

= — E ( (QX(s),X(s))ds. 
Jo 



(3.7) 



This implies 

El (QX(s),X(s))ds = (Px,x) -E(PX(t),X(t)) < (Px,x), t>0. (3.8) 



Thus, the system is Lg-globally integrable. 
(i) (iii) is clear. 

(iii) => (i). Suppose (3.4) has a solution P £ S n . Then by (3.8), we have 

t 

(QX(s),X(s))ds < (Px,x)+LE\X(t)\ 2 . 

Hence, [A, C] is Lg-globally integrable. □ 
Combining Propositions 3.3 and 3.5, we have the following result for system [A, C]. 

Proposition 3.6 The following arc equivalent: 

(i) System [A,C] is L 2 -exponentially stable; 

(ii) System [A, C] is L 2 -globally integrable; 

(iii) For any Q > 0, the Lyapunov equation (3.4) admits a solution P > 0, and in this case, the 
representation (3.5) holds for this P; 

(iv) System [A, C] is L 2 -asymptotically stable, and for some Q > 0, Lyapunov equation (3.4) admits a 
solution P G S n . 
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Proof. The implications (i) (ii) => is clear. The relations (ii) <S=> (iii) •<=>• (iv) follow from 
Proposition 3.5. The implication (iii) =^ (i) follows from (3.7), together with the positive defmitencss of P 
and Q and Gronwall's inequality. □ 

Now, let us return system [A, A, C, C]. We have the following result. 

Theorem 3.7 (i) Suppose system [A, A, C, C] is L 2 -asymptotically stable. Then it is necessary that A + A 
is exponentially stable. 

(ii) If A + A is exponentially stable, then system [A, A, C, C] is L 2 -exponentially stable if either [A, C] is 
L? -globally integrable, or 

C + C = 0. (3.9) 
Proof, (i) Suppose (3.2) holds. Taking expectation in (3.1), we obtain 

(dE[X(t)] = (A + A)E[X{t)]dt, t>0, 
\E[X(0)]=s. 

Hence, 

E[X(t)] = e^ A+A ^x, t > 0. 

Since 

|E[X(t)]| 2 < E\X(t)\ 2 , t>0, 
the I/ 2 -asymptotic stability of system [A, A, C, C] implies the exponential stability of A + A. 
(ii) By (2.4) with B = B = D = D = 0,we have, for any P £ <S'\ 

E(P{X(t) -E[X(t)]},X(t) -E[X(t)]) 

, 1^ { ((PA + A T P + C T PC){X(s) -E[X(s)]},X(s) -E[X(s)}) (3.11) 

+ ( P(C + C)E[X(s)}, (C + C)E[X(s)] ) }ds. 

Hence, if (3.9) holds, one has from the above that 

var [X(t)} < L [ var [X(s))ds, Vt > 0. 
Jo 

Then, by Gronwall's inequality, we obtain 

var[X(i)] = 0, <>0. 
Consequently, if we let 2A = — maxer(^4 + A) > 0, then 

e 2Xt E\X(t)\ 2 = e 2At (var[X(i)] + |E[X(i)]| 2 ) = \e xt e {A+A)t x\ 2 -> 0, t -»• oo. 

Thus, [A, A, C, C] is L 2 -exponentially stable. 

Next, if [^4,C] is L 2 -globally integrable, then by Proposition 3.6, for Q = I, there exists a P > such 
that 

PA + A T P + C T PC + 1 = 0. 

Hence, (3.11) implies 

vax[X{t)]<-fx var[X(s)}ds + L \E[X (s)}\ 2 ds , t>0, 
Jo Jo 
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for some fi, L > 0, with fj, ^ A = — ma.xa(A + A) > 0. By GronwaH's inequality. 

vax[X(t)] < L\x\ 2 I 
Jo 



-At _ — /it 

^ t - s ) e - As ds = Lb| 2 - , t>0. 

[i — A 



This results in 



>e -At _ e - M i 



E|X(t)| 2 = var [X(t)] + |E[X(t)]| < L|x| 2 - - + | e ( A+ ^*a;| 2 , t > 0. 

fi — A 

Therefore, the system [A, A, C, C] is L 2 -cxponentially stable. This completes the proof. □ 

Note that the exponential stability of A + A together with the L 2 -global integrability of [A, C] or (3.9) 
are sufficient conditions for the L 2 -exponential stability of system [A, A, C, 0]. When n = 1, these conditions 
are also necessary in some sense. To be more precise, let us look at the following one-dimensional system: 



j dX(t) = {aX(t)+aE[X(t)]}dt + {cX{t) + cE[X(t)}}dW{t), t > 0, 
X(0) =x. 



We have the following result. 

Proposition 3.8 For system (3.12), the following arc equivalent: 

(i) It is L 2 -exponentially stable; 

(ii) It is L 2 -globally intcgrablc; 

(iii) It is L 2 -asymptotically stable; 

(iv) a + a < 0, and 

either 2a + c 2 < 0, or 2a + c 2 > and c + c = 0. 

Proof. It suffices to prove the implication (iii)=>(iv). By (2.3) with P = 1, B = B = D = D = 0, A = a, 

A = a, C = c, C = c, we have 



E\X(t)\ 2 = .t 2 +E J \{2a + c 2 )\X{s)\ 2 + (2a + c 2 + 2cc)(E[X(s)]) 2 }ds 

+ I {(2a + c 2 )E|X(s)| 2 + [25 - c 2 + (c + c) 2 ]x 2 e 2(a+a)s }ds. 



x 2 



Thus, 

E\X(t)\ 2 = e^+^x 2 + [2a-c 2 + (c + c) 2 ]x 2 I e (^)(ts) e 2(a+a)s ds 



II 



e ( - 2a+c ^x 2 + (2a-c 2 )x 2 e ( - 2a+c ^ t / e^ c >ds + (c + c) V / e (2a+c-)(t-) e 2(a+a). dfl 

Jo Jo 

e (2a+c 2 )t ;r 2 + ^a+c^t [ e (2a-e 2 )t _ l] + ( c + g) V /' e (^)(t~s) e 2(a + a)s dg 
x 2 e 2(a+a)t + (fi + ^2 e (2a +C 2 )(t- ,) e 2(a+a) S ^ 







JO 

Now, if (3.2) holds, then we must have 

a + a < 0, 

and 

(C + C) 2 /' e (2 a + C 2 )(t- S ) e 2(a+a) Sds ^ q 
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Thus, under a + a < 0, ifc + c^O, then we need 

,(2a+c 2 Kt-s) e 2(a+a)s ds = e (2a+c 2 ) t / gpB-c*).^ ^ Q _ 



Since J * e^ 2 ° c2 ' s ds is increasing, the above must lead to 2a + c 2 < 0. Also, if 2a + c 2 > 0, we must have 
c + c = 0. This completes the proof. □ 

Now, for the L?, ^-global integrability of system [A, A, C, CI, we have the following result. 

Proposition 3.9 Let (J) hold. If [A,A,C,C] is L 2 ^ ^-globally integrable, then A + A is L^^-globally 
intcgrable, i.e., 

/>oo 

\(Q + Q)h {A+A)t \ 2 dt < oo. (3.13) 



Conversely, if (3.13) hold, then [A, A, C, C] is Lq Q-globally integrable provided either (3.9) holds, or [A, C] 
is Lg-globally integrable and 

M(Q + Q)CM(C + C), (3.14) 

where Af(G) is the null space of G. 
Proof. Since, 

(03 + Q)E[X(t)],E[X(t)] )dt<E^ ( ( QX(t),X(t) ) + ( QE[X(t)],E[X(t)] ) )<ft < oo, 

we see that (3.13) follows. 

Next, let (3.13) hold. If (3.9) holds, we have (see (3.11) with P = I) 

var [X(t)\ = E ( {X(t) - E[X (t)]} , X (t) - E[X(t)] ) 

= eJ* { ((A + A T + C T C){X{s) - E[X(s)}},X(s) - E[X(s)} ) +|(C + C)E[X( s )]| 2 }rf.s 

= E [ ((A + A T + C T C){X(s) - E[X(s)}}, X(s) - E[X(s)) ) ds < L [ var [X(s)]ds. 
Jo Jo 

Hence, by Gronwall's inequality, we obtain 

var [X (i)] = 0, t > 0. 

Consequently, 

E / ( (QX(t),X(t)) + (QE[X(t),E[X(t)])yt 

[Q{X(t) - E[X(t)},X(t) - E[X{t)} ) + ({Q + Q)E[X(t)],E[X(t)} >)di 
< J™ (|Q|var [X(t)} + \(Q + Q)i e^+^x^dt 
\(Q + Q)hS A+A ^x\ 2 dt<oo, 



which gives the Lq ^-global integrability. 
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Finally, if (3.13) holds and [A, C] is Lg-globally integrable, then by Proposition 3.5, we can find a P > 
solving Lyapunov equation (3.4). Let X(-) be the solution of (3.1). Applying Ito's formula to ( PX(-), X(-) }, 
we get 

E{P{X(t) -E[X(t)}},X(t) -E[X(t)}) 

= E f*{ (i PA + ATp + C T PC){X(s) - E[X(s)]},X(s) - E[X(s)] ) 

+ {{C + C) T P(C + C)E[X(s)],E[X(s)} ) \da 
= E J o {-( Q{ x ( s ) -^[X(s)]},X(s)-E[X(s)] ) + ((C+C) T P(C+C)E[X(s)l E[X(s)} )}ds. 

Now, condition (3.14) implies that 

(P(C + C)y,(C + C)y) < L{(Q + Q)y,y), Vyel", 

for some L > 0. Thus, 

E f (Q{X(s)-E[X(s)]},X(s)-E[X(s)])ds 
Jo 

t 

(P(C + C)E[X(s)],{C + C)E[X(s)])ds-E(P{X(t) -E[X(t)]},X{t) -E[X(t)}) 

<L [ ((Q + Q)E[X(s)],E[X(s)])ds. 
Jo 

Consequently, 

E J ((QX(t),X(t)) + ( QE[X(t)] , E[X(t)] ) ) dt 

- j ((Q{X(t)-E[X(t)]},X(t)-E[X{t)}) + ((Q + Q)E[X{t)],E[X(t)})yt 

< (L + 1) / ((Q + Q)E[X(s)],E[X(s)} ) ds < oo. 
Jo 

This means that the system is ^-globally integrable. □ 
We point out that condition (3.14) holds if (3.9) is true or 

Q + >0. 

Therefore, to have condition (3.14), we do not have to assume (J)'. 

4 MF-Stabilizability 

We now return to the controlled linear MF-FSDE (1.1) which is denoted by [A, A, C, C; B, B, D, D]. With 
this notation, we see that the uncontrolled MF-FSDE (3.1) is nothing but [A, A, C, C; 0, 0, 0, 0]. Note also 
that in the case A = C = and B = D = 0, the system is a usual controlled linear SDE, which is simply 
denoted by [A, C; B, D] = [A, 0, C, 0; B, 0, D, 0]. Further, in the case C — and D = 0, the system is reduced 
to a classical controlled linear ODE, which is denoted by [A; B] = [A, 0, 0, 0; B, 0, 0, 0]. We now introduce 
the following notion for general state equation (1.1). 
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Definition 4.1 (i) Let (J) hold. System [A, A, C, C; B, B, D, D] is said to be MF-L 2 Q q- stabilizable if there 
exists a pair (K,K) e R nXm x R" xm such that for any x e R" if X K < R (-) is the solution to the following: 

' dX K ' R (t) = {(A + BK)X K ' R {t) + [A + BK + B(K - K)]E[X K < R (t)]}dt 

+ {{C + DK)X K ' R (t) + [C + DK + D{K-K)]E[X K ' R {t)]}dW(t), t>0, 

,X K > R (0)=x, 



and 
then 



.K.K i 



3 



(t) = K{X K K {t) - E[X K > K (t)}} + KE[X K - K (t)}, t > 0, 



; QX K > R (t), X K > R {t) ) + ( QE[X R ' R (t)),E[X K > R (t)} ) + \u K > R (t)\ 2 ) dt < 



oo. 



(4.1) 



(4.2) 



In this case, the pair (K, K) is called an MF-L^ Q-stabilizer of the system. In the case that (4.2) is replaced 
by the following: 

^ (\X K > R (t)\ 2 + \u K - R (t)\ 2 )dt<oc, 

we simply say that the system is MF-L 2 -stabilizable, and (K, K) is called an MF-L 2 -stabilizcr of the system. 

(ii) Let (J) hold. System [A, A, C, C; B, B, D, D] is said to be L 2 Q q- stabilizable if there exists a K £ R nxm 
such that for any x 6 R™, if X K (■) is the solution to the following: 



>dt 



' dX K {t) = [{A + BK)X K (t) + {A + BK)E[X K (t)]}c 

+ {(C + DK)X K (t) + (C + DK)E[X K (t)]}dW{t), t > 0, 

.X K (0) = x, 



(4.3) 



and 



then 



u K (t) = KX K {t), t>0, 



3 



QX (t), X K (t)) + ( QE[X {t)\, E[X K (*)] ) +\u K {t)\ 2 )dt < oo 



(4.4) 



In this case, K is called an L 2 ^ Q-stabilizer of the system. In the case that Q = 0, we simply say that the 
system is Lq- stabilizable, and K is called an Lq- stabilizer. If (4.4) is replaced by 

EjT (\x K (t)\ 2 + \u K {t)\ 2 yt < oo, 

we further simply say that the system is L 2 - stabilizable, and K is called an L 2 -stabilizer of the system. 

The importance of the notions defined in the above definition is that if (J) holds and [A, A,C,C; B, B, D, D] 
is MF-Lq ^-stabilizable, then U a d[0, oo) is nonempty since u K - K {-) defined by (4.1) is in U a d[0, oo). ln par- 
ticular, U a d[0, oo) is nonempty if the system [A, A, C, C; B, B, D, D] is MF-L 2 -stabilizable. 

It is seen that when system [A, A,C,C; B, B, D, D] is MF-Lq ^-stabilizable, then the uncontrolled system 
[A + BK, A + BK + B(K - K), C + DK, C + DK + D(K - K)\ is L 2 ..-globally integrable. Also, system 
[A, A, C, C; B, B, D, D] is Ln-stabilizable if and only if 



(QX K (t),X K (t))+\u K (t)\ 2 )dt < oo 
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Moreover, it is clear that the L 2 -stabilizability of system [A, C; B, D] wc defined here is the classic stabiliz- 
ability of the controlled SDE system. 

Note that system (1.1) is Lq g-stabilizable (resp. L 2 -stabilizability) if it is MF-Lg g-stabilizable (resp. 
MF-L 2 -stabilizability) with K = K. Therefore, the former is a special case of the later. The following 
example shows that in general, the MF-L 2 -stabilizability does not imply the L 2 -stabilizability. 

Example 4.2 Consider the following one-dimensional controlled MF-FSDE: 

dX{t) = {aX{t) + oM[X(t)} + bu(t) + bE[u(t)]}dt 

+ {cX(t) + cE[X(t)] +du(t) +M[u(t)]}dW(t), t > 0, 
[X(0) = x. 

Suppose the above system is MF-L 2 -stabilizable. Then, there are k, k £ K such that with 

u(t) = k{X(t) - E[X(t)}} + kE[X(t)\, t > 0, 

the closed-loop system: 

dX(t) = {(a + bk)X(t) + [d + bk + b(k-k)}E[X{t)]}dt 

+ {(c + dk)X(t) + [c + dk + d(k - k)]E[X(t)]}dW(t), t > 0, 

is L 2 -globally intcgrablc. By Proposition 3.8, this is equivalent to the following: 

a + a + {b + b)k < 0, 

and either 

2 



2(a + bk) + (c + dky <0 



or 



2(a + bk) + (c + dk) 2 > 0, c + c + (d + d)k = 0. 

Let 

6 + 6 = 1, d=l, d=-l, c + c^0. 

Then we need and only need 

a + d + k = -\<0, 2(a + bk) + (c + k) 2 < 0, (4.5) 

for some k, k £ R. The first condition in (4.5) can always be achieved. The second one is equivalent to the 
following: 

> k 2 + 2(6 + c)k + 2a + c 2 = (k + b + c) 2 + 2a + c 2 - (6 + c) 2 , 

which is possible if 

2a + c 2 - (6 + c) 2 < 0. (4.6) 
On the other hand, in order the system to be stabilizable, we need k = k, and 

> 2(a + bk) + (c + kf = 2 [a - b(a + a + A)] + [c - (a + a + A)] 2 
= A 2 - 2(a + d + b - c)\ + (a + d - c) 2 + 2 [a - b(a + a)], 

for some A > 0. This is impossible if, say, 

c > a + a + 6, a - b(a + a) > 0. (4.7) 

It is easy to find cases that (4.6)-(4.7) hold. Hence, we see that MF-L 2 -stabilizability does not imply 
L 2 -stabilizability, in general. 
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Now, we present a result concerning the MF-Li g-stabilizability of system (1.1). 



Theorem 4.3 Let (J) hold. 

(i) If system (1.1) is MF-Lq Q-stabilizable, then the controlled ODE system [A + A; B + B] is Lq + q~ 
stabilizable, i.e., for some K £ R mxn , 

/ \(Q + Q)^e^ A+A+ ^ B+ ^ R ^dt<oo. (4.8) 
Jo 

(ii) Suppose the following holds for some K £ R mx ™ satisfying (4.8): 

N{Q + Q) C M{C + C) + (D + D)K) . (4.9) 

Further, suppose the controlled SDE system [A,C;B,D] is Lq-stabilizable. Then the controlled MF-FSDE 
system [A, A, C, C; B, B, D, D] is MF-L 2 Q ^-stabilizable. 

(iii) Suppose the following holds for some K € K" xm satisfying (4.8): 

C + C + (D + D)K = 0. (4.10) 

Then the controlled MF-FSDE system [A, A, C, C; B, B, D, D] is MF-L 2 Q Q -stabilizable. 

Proof. Under (4.1), the closed-loop system takes form (4.3). According to Proposition 3.9, we know that 
if (4.3) is ^-globally integrable, it is necessary that (4.8) holds, which proves (i). Further, when (4.8) 
holds, the system (4.3) is Lg g-globally integrable if either the system [A + BK, C + DK] is stable and (4.9) 
holds, which proves (ii), or (4.10) holds with the same K which proves (iii). □ 

The above leads to the following corollary. 

Corollary 4.4 (i) If system (1.1) is MF-L 2 -stabilizable, then the controlled ODE system [A + A; B + B] is 
stabilizable, i.e., there exists a K £ R nxm such that 

a(A + A+ (B + B)K) C C~. (4.11) 

(ii) Suppose controlled ODE system [A + A; B+B) is stabilizable, and controlled SDE system [A, C;B,D] 
is I? -stabilizable. Then the controlled MF-FSDE system [A, A, C, C; B, B, D, D] is MF-L 2 -stabilizable. 

(iii) Suppose (4.10) holds for some K £ R" x " i satisfying (4.11). Then the controlled MF-FSDE system 
[A, A, C, C; B, B, D, D] is MF-L 2 -stabilizable. 

Note that conditions assumed in (ii) of Corollary 4.4 do not involve C and D. However, condition (4.10) 
involves both C and D. We point out that (4.10) means that 

U(C + C) C K(D + D). (4.12) 

In the case that m < n, the above could be a big restriction on C + C and D + D. Moreover, we have to 
find the same K £ R mxn such that (4.11) and (4.10) hold at the same time. If we let (D + D)+ be the 
Moore- Penrose pseudo-inverse of D + D ([7]), then the solution of (4.10) is given by 

K = -(£> + D)+(C + C) + [I - (D + D)+(D + D)] K, 

for some K £ M mxn . Thus, we need 

a(A + A + {B + B){ -(D + D)+(C + C)+ [I -{D + D)+(D + D)]k}^ CC" 
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for some K € R mxn , which means the ODE system 

A + A- (B + B)(D + D)+(C + C); (B + B)[l - {D + D)+ [D + D)]\ (4.13) 
is stabilizable. Hence, we obtain the following result. 

Proposition 4.5 Let (4.12) hold. Then [A, A, C, C; B, B, D, D] is MF-L 2 -stabilizable if ODE system (4.13) 
is stabilizable, which is the case, if, in particular, m = n, D + D is invertiblc, and 

a(A + A-(B + B)(D + D)- 1 (C + C)) C CT. (4.14) 

Condition (4.14) seems that the MF-L 2 -stabilizability of system [A, A, C, C; B, B, D, D] could be nothing 
to do with the stabilizability of the controlled linear SDE system [A, C; B, D]. However, in the case that 
A = C = and B = D = 0, we have the following controlled linear SDE: 

dX(t) = ^AX{t) + Bu(t)}dt + \(JX{t) + Du(t)}dW(t), t > 0. 

Suppose m = n and D~ l exists. Then condition (4.14) becomes 

a(A-BD~ 1 C) CC", (4.15) 

In this case, if we take 

u(t) = -D~ 1 CX{t), t > 0, 

then the closed-loop system becomes 

dX(t) = (A-BD- 1 C)X(t)dt, t>0, 
which is stable if (4.15) holds. Interestingly, if we let 

u(t) = -D- r C^[X{t)\, Vt > 0, (4.16) 

then the closed-loop system reads 

dX{t) = ^AX{t) - BD^CElX^jldt + c{x{t) - E[X(t)]jdW(t), t > 0, 
X(0) = x. 

It is not hard to see that the unique solution X(-) of the above is deterministic and given by 

X{t) = e( A - BD ~ lc »x, t > 0. 
Therefore the system is also asymptotically stable under feedback control (4.16). 

5 Stochastic LQ Problems 

In this section, we study a classic stochastic LQ problem, which will be crucial for Problem (MF-LQ). We 
consider the following controlled SDE: 

dX(t) = ^AX{t) + Bu(t)}dt + \cX{t) + Du{t)}dW{t), t > 0, 
X(0) = x, 
and cost functional 

poo 

J°(x; u(-)) = E / { ( QX(t), X(t) ) + { Ru(t),u{t) ) }dt. 
Jo 

Let 



X^ d [0,oo) = {X(-) e ^ oc [0,oo)|E^ (QX{t),X(t))dt < oo}, 
Uadl ? 00 ) = {«(•) eW[0,oo) X(x;u(-)) E X® d [Q, oo), Viel"}. 
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5.1 A Classic Stochastic LQ Problem 

We introduce the following assumptions. 

(J)* The matrices Q e S n and R G S m satisfy 

Q>0, R>0. 

(S)* The system [A,C;B,D] is L^-stabilizable. 
Let us pose the following problem. 

Problem (LQ). For any x G K™, find a u„(-) G WfjO, oo) such that 

J°(a:;u*(-)) = inf J°(x;u(-)) = V°{x). 
u(-)ew^j[o,oo) 

We have the following result. 

Theorem 5.1 Let (J)* and (S)* hoid. Then Problem (LQ) admits a unique optimal control uq(-) G 
Z^[0,oo). Mor eover, the following ARE admits a solution P > 0: 

PA + A T P + C T PC + Q- {PB + C T PD){R + D T PD)- 1 (B T P + D T PC) = 0, 
and r is an Lg-stabilizcr of [A, C; B, D], where 

r = -(R + D T PDy 1 (B T P + D T PC). (5.1) 
Further, the optimal control uq(-) is given by 

u Q (t) =TX Q (t), t>0, 
with the optimal state process X®(-) being the solution of closed-loop system: 

f dX Q (t) = (A + BT)X Q (t)dt + (C + DT)X Q (t)dW(t), t > 0, 

\xQ(o) = x, 

and 

(Px,x)= inf J°{x;u{-)) = V°(x), Vx G R™. (5.2) 
«(-)ew a Q d [o,oo) 

Proof. First of all, it is clear that under (J)* and (S)*, the set U® d [0,oo) is nonempty, and (x,u(-)) h-» 
J°{x; u(-)) is a quadratic functional, coercive with respect to u(-) G Z^[0, oo). Thus for any x G K", there 
exists a unique optimal control m^(-) G Z//^[0, oo), and the value function x i-> V (x) must be of form (5.2) 
for some P > 0. We now would like to determine P and the optimal pair (X*(-), «*(•)). To this end, let us 
introduce 

J^(x;u(-))=E [ {(QX(t) > X(t)) + (Ru(t),u(t))}dt, T > 0, 
Jo 

where it(-) G £Y/ OC [0, oo) and X(-) = «(•)). It is standard that under (J)*, there exists a unique 

u$(-) e«[0,T] such that 

V#(a;) = inf jS(ac:«(0) = J&(a:;uS(0) = (P(0; !>,«), VieM", 
u(-)eu[o,T] 
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with P(- ; T) being the solution to the following differential Riccati equation: 

P(t; T) + P(t; T)A + A T P{t; T) + C T P(t; T)C + Q 

-[P(t;T)B+C T P(t;T)D] [R+D T P(t; T)D] _1 [B T P(t; T) + D T P(t; T)C] = 0, t G [0,T], 
P(T;T) = 0. 

Moreover, the optimal control mt(-) can be represented as follows: 

u$(t)=T(t;T)X$(t), te[0,T\, 

with 

r(t;T) = - [R + D T P{t;T)DY 1 [B T P{t;T) + D T P(t;T)C], t G [0,T], 
and Xj!(-) is the solution to the following closed- loop system: 

f dX®{t) = [A + BT(t;T)]X$(t)dt+ [C + DT{t;T)}X®{t)dW{t), t G [0,T], 
I X«(0)=x. 

Now, it is clear that 

Jr(^O) < Jffa O), Vw(-) G W[0, T], 0<T<T<oo. 

Therefore, one has 

< P(0; T) < P(0; f ), V0 < T < f < oo. 

On the other hand, since 

U^T] = {u(-)\ [0T] | 4)ew a Q J0,»)}cw[o,r], 

it is true that 

(P(0;T)x,x) = V°(x) = inf jS.(x; u(-)) < inf J%(x;u(-)) 

n(-)eU[0,T] u(-)SWf d [0,T] 

< inf J°(x;u(-))=V°{x) = (Px,x), Vxef". 
u(-)ewf d [o,oo) 

Combining the above, we see that 

< P(0; T) < P(0; T) < P, V0 < T < f < oo. 

This implies that 

lim P(0; T) = P < P, 

T— >-oo 

for some P(-) > 0. Now, we introduce the following differential Riccati equation (on [0, oo)): 

-P(s) + P(s)A + A T P(s) + C T P(s)C + Q 

- [P{s)B + C T P(s)D] [R + D T P(s)D] -1 [B T P{s) + D T P(s)C] = 0, s > 0, 
P(0) = 0. 

For any T > 0, let 

?{t;T) = P(T-t), te[0,T]. 
Then by the uniqueness, we must have 

P(t;T) = P{t;T) = P(T-t), <e[0,T]. 
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Hence, 

P(0;T) = P(T), T>0. 



From (5.5), we have 



lim P(t) = P. 

t— >oo 



This P > must be a solution to the algebraic Riccati equation: 

PA + A T P + C T PC - {PB + C T PD)(R + D T PD)- 1 (B T P + D T PC) + Q = 0. 
Further, from (5.6), one has 



Consequently, 



lim P(t; T) = lim P(T -t) = P, t > 0. 

oo 



lim r(t;T) = -(R+ D T PD)~ 1 (B T P + D T PC) =T Q , \/t>0. 

T—foo 



Note that (suppressing (t;T)) 

P(A + BT) + (A + BT) T P +(C + DT) T P(C + DT) + T T RT 

= P[A- B(R + D T PD)- 1 (B T P + D T PC)] + [A - B(R + D T PD)- 1 (B T P + D T PC)] T P 
+ [C-D(R+ D T PD)- 1 (B T P + D T PC)] T P [C - D(R + D T PD)- 1 {B T P + D T PC)] 
+ (PB + CPD T )(R + D T PD)- 1 R(R + D T PD)- 1 (B T P + D 7 ' PC) 

= PA + A T P + C T PC - PB(R + D T PD)- 1 (B T P + D T PC) 

-{PB + C T PD)(R + D T PD)- 1 B T P - {PB + C T PD){R + D T PD)- l D T PC 
-C T PD{R + D T D)- 1 {B T P + D T PC) 

+ {PB + C T PD)(R + D T PD)- 1 D T PD(R + D T PD)- 1 (B T P + D T PC) 

+ (PB + CPD T )(R + D T PD)- 1 R(R + D T PD)- 1 (B T P + D T PC) 
= PA + A T P + C T PC - (PB + C T PD)(R + D T PD)- 1 (B T P + D T PC) 

-{PB + C T PD)(R + D T PD)- 1 (B T P + D T PC) 

+ {PB + CPD T )(R + D T PD)- 1 (B T P + D T PC) 
= PA + A T P + C T PC - (PB + C T PD)(R + D T PD)- 1 (B T P + D T PC). 

Next, we rewrite the differential Riccati equation (5.3) as follows: 

P(t; T) + P{t- T) [A + BT{t- T)] + [A + BT(t; T)] T P{t; T) 

+ [C+Dr(t;T)] T P(t;T)[C+Dr(t:T)]+r(t;T) T Rr(t;T)+ Q= 0, te [0,T], 
P(T;T) = 0. 



It is clear that (see (5.4) and (5.7)) 



lim X$(t) =X Q {t), t> 0, 



T-too 



with X^(-) being the solution to the following: 



f dX Q (t) = (A + BT )X Q (t)dt + {C + DT )X Q (t)dW(t), t > 0, 

lx(o) = o. 
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Further, 



; P(0; T)x, x ) = ~eJ {({P(t; T)+P(t; T) [A+BT(t; T)] + [A(t; T) + BT(t; T)] T P(t; T) 
+ [C+DT(t; T)] T P(t; T) [C+DT(t; T)] }X#(t), X$(t) )\dt 

= E / ( [Q + r(i;T) T i?r(t;T)]X^),X^))cft 
Jo 



( QA^ (t) , (t) } + { J2T(i; T)X^(i), r(t; T)x£ (t) ) ) di 



Thus, by Fatou's Lemma, we obtain (see also (5.5)) 

(Px,x) > (Px,x) > E^ ((QX Q (t),X Q {t)) + (RT X Q (t),T X Q {t))^dt > V°{x) = (Px,x), 



which implies 



p = p, r = r, 



and X Q (-) e X2i[0,oo). Also, T defined by (5.1) is an L|-stabilizer of [A, C;B,D]. This completes the 
proof. □ 

5.2 Stochastic MF-LQ Problem 

Having the above, let us now return to Problem (MF-LQ). We introduce the following assumption. 

(S) The controlled ODE system [A + A; B + B] is stabilizable, and the controlled SDE system [A, C; B, D] 
is L 2 -stabilizable. 

From Corollary 4.4, we know that under (J) and (S), the system [A, A, C, C; B, B, D, D] is MF-L 2 - 
stabilizablc. We point out that it is possible for us to relax (S) in various ways. However, for the simplicity 
of presentation, we would like to keep the above (S). Let us first present the following result. 

Now, we are ready to state and prove the main result of this paper. 

Theorem 5.2 Let (J) and (S) hold. Then Problem (MF-LQ) admits a unique optimal control £ 
U a d[0, oo), and the following AREs: 



PA+A T P+C T PC+Q-{PB+C T PD){R + D T PDy 1 {B T P+D T PC) = 0, 
U(A+A) + (A+A) T U+(C+C) T P(C+C)+Q+Q 

- [U(B + B) + (C+C) T P(D + D)] [R + R+(D + D) T P(D + D)] _1 
■[{B + B) T U + (D + D) T P{C+C)] = 0, 

admits a solution pair (P, II) G 5™ x <S?. Define 

T = -{R + D T PDy 1 (B T P + D T PC), 

p_ r E> I D I l n I t\\Ttd!t^ I f»VI lYD I d\7 , tt I I n I r>\T 



(5. 



[R + R + (D + D) T P{D + D)] [{B + B) T U + (D + D) T P(C + C)] . 

2 

<2,<2~ 

dX*(t) = {(A + BT)X,(t) + [A + BT + B(t - T)]W,[X*(t)]^dt 



Then (T,T) is an MF-L' 2 ^-stabilizer of the system. If X*(-) is the solution to the following MF-FSDE: 



-{(C + DT)X*(t) + [C + DT + D(T - T)]M[X,(t)]}dW(t), t > 0, 



{ X*(0) = x, 
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then 



inf J(x:u(-)) = J(x: u*(-)) = (Hx,x ), Vx £ 

u(-)€U ad [0,oo) 



(5.9) 



with the optimal control it*(-) £ U a d[0, oo) admits the following state feedback representation: 

u*(t) = T{X*(t) - E[X*(t)]} + fE[X*(t)], t > 0. 

Proof. We know that under (J) and (S), the set lA ac i[0,oo) is nonempty, and convex. For any (x,u(-)) £ 
M. n x Uad[0,oo), let X(-) = X{-;x,u(-)) £ X[0,oo). Then J(x;u(-)) is well-defined and 



J(x; u(-)) = EJ {{QX(t),X(t) )+( QE[X(t)],E[X(t)] } + { Ru(t), u(t) )+( RE[u{t)], E[u(i)] } }d< 
^' { { Q{A-(t) - E[X(t)]},X(i) - E[X(i)] > + <(Q + Q)E[X(i)],E[X(i)] ) 
+ ( R{u(t) - E[u(t)) } , u(t) - E[u(t)} ) + ((R + R)E[u(t)) , E[u(t)} ) Xc 

pOO 

>SE \u(t)\ 2 dt, 



>dt 



for some 6 > 0. Therefore, under (J) and (S), the map u(-) i— > J(x; u(-)) is a quadratic and coercive functional 
on U a d[0, oo). Hence, by a standard argument, we see that optimal control «*(•) £ U a d[0, oo) must uniquely 
exist, and of course, X t (-) is also unique. By a standard argument, we can show that value function V(x) is 
of form (5.9) for some II £ S n , U > 0. 

Now, for any T > 0, let 

l-T, 



J T (x; «(•)) = E y { ( * (*) )+( QE[X(t)},E[X(t)} }+(Ru(t),u(t) }+(RE[u(t)],E[u(t)] ) } 

We may pose the following problem. 

Problem (LQ) T . For any x £ R n , find a u T (-) £ U[0,T] such that 

J T (x;u T (-)) = inf J T (x; u(-)) = V T (x). 
u(-)eU[o,T] 

By [35], for Problem (LQ)t, under (J), we have a unique ut(-) £ U[Q,T] such that 

V T (x)= inf J t (x]u(-)) = J t (x;ut(-)) = (U(0;T)x,x), Vx 6 R™, 

ti(-)GW[0,T] 



where 



P(t; T) + P(t; T)A + A T P(t; T) + C T P(t; T)C + Q 

-[P(t;T)B+C T P{t;T)D] [i?+L> T P(i; T)L>] _1 [B T P(t;T) + D T P(t;T)C] = 0, t £ [0,T] 
P(T;T) = 0, 



(5.10) 



and 



U(t- 1 T)+n(t;T)(A+A) + (A+A) T U(t;T) + (C+C) T P(t;T)(C+C) + Q + Q 

- [U(t; T){B + B) + {C+C) T P{t- T)(D + D)] [R + R+(D + D) T 'P(t; T)(D + D)] ^ 
■[{B + B) T U(t;T) + {D + D) T P(t;T)(C+C)] = 0, t £ [0,T], 
[IL(T;T) = 0. 
Further, if we define 

T(t; T) = -[R+ D T P(t; T)D] _1 [B T P(t; T) + D T P(t\ T)C] , 

f (t; T) = -[R+R+(D + D) T P(t; T)(D + D)] _1 [(B + B) T P(t; T) + (D + D) T P(t T){C+C)} , 
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then the optimal control ut(-) admits the following state feedback representation: 

u T (t) = T(t; T){X T {t) - E[X r (t)]} + f (f; T)E[X T (t)}, t e [0, T], 
where Xt(-) is the solution to the closed- loop system: 

dX T (t) = { [A + BT(t; T)] X T (t) + [A + Bf(t; T) + B(f(t; T) - T(t; T))]K[X T (t)]}dt 

+ { [C + DT{t; T)]X T (t) +[C + DT(t; T) + D(f (t; T) - T(t; T))]E[X T (t)]}dW(t), 

te[o,T], 

{ X T (0) = x. 

Observe that (5.10) coincides with (5.3). By the proof of Theorem 5.1, we see that 



Hence, 



lim P(t; T) = P, t>0. 



lim T(t;T) = -(R + D T PD)- 1 (B T P + D T PC) = T, t>0. 

T— >oo 



Now, we introduce the following differential Riccati equation (on [0,oo)): 

-ft(s) + U(s)(A+A) + (A + A) T U(s) + (C+CfP(s)(C+C) + Q + Q 
- [U(s)(B + B) + (C+C) T P(s)(D + D)] [R + R +(D + D) T P(s)(D + D)] ~* 
■[(B + B) T Tl(s) + {D + D) T P(s)(C+C)] =0, t > 0, 

[n(o) = o. 

For any T > 0, let 

H(t;T) =n(T-t), ie[0,T]. 
Then by the uniqueness, we must have 

n(t;T) =fi(t;T) =H{T-t), t£[0,T]. 

Hence, 

n(0;T) = n(r), t>o. 

Similar to the proof of Theorem 5.1, we have that 

< n(0; T) < 11(0; f) < n, < T < f < oo. 

Thus, 

lim fl(t) = n < n. 

t— >oo 

Further, ft must be a solution to the following ARE: 

U(A + A) + (A + AfU +(C + C) T P(C + C) + Q + Q 

- [U(B + B) + (C + C) T P(D + D)] [R + R + (D + D) T P(D + D)] _1 
■[(B + B) T U+(D + D) T P{C+C)] =0, 



Also, 



lim lift: T) = lim U(T -t)=IL, t > 0. 

T~>oo T->oo 



Then 



(5.11) 



lim r(i; T) = T = - [R + R + (D + D) T P(D + D)\ 1 \{B + B) T fl + (D + D) T P(C + C)l , Vi > 0. 
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Recall that Xt(-) satisfies (5.11). Thus, one has 

lim X T (t;T) = X(t), t > 0, 

with X(-) being the solution to the following: 

dX{t) = {(A + BT)X{t) +[A + Bf + B(G - T)]E[X(t)]\dt 

+{(c + DT)x(t) + [c + Dr + D(T -r)]E[x(t)]}dW(t), te[o,T\, 

K X(0) = x. 
On the other hand, 

(U(0;T)x,x) = J T (x;u T (-)) 

: J { ( QX T (t) ,X T (t)) + ( QE[X T (t)} , E[X T (t)} > 

+ (i?(r(t;T){X T (t)-E[X T (t)]}+f(t;r)E[X T (t)]),r(t;T){X T W-E[X T (t)]}+f(i;T)E[X T (0]) 
+ ( RT(t; T)E[X T {t)\, f (t; T)E[X T (t)} ) }dt. 

Thus, sending T — > oo, by Fatou's Lemma, we obtain 

(nx,x) > (Ux,x) >E J I (QX(t),X(t)) + (QE[X(t)},E[X(t)}) 

+(R(T{X(t)-E[X(t)]}+f E[X(t)]),T{X(t)-E[X(t)}}+foE[X(t))) 
+ {RT E[X(t)],t E[X(t)}) }dt = J(x;u(-)) > (ILc,x). 

Hence, 

n = n, f o = f , 

and (r, f) is an MF-L^ ^-stabilizer of the system, and (X(-), u(-)) = «*(■)) is the optimal pair. □ 

6 Optimal MF-LQ Controls Presented via Tackling AREs 
6.1 Tackling AREs via LMIs 

One of the main ideas of this section is to reformulate the AREs as linear matrix inequalities (LMIs, for 
short). Let us introduce the general notion of LMIs according to [1, 27], and develop it to solve our mean- field 
LQ problem. 

Definition 6.1 Let i*o,-Fi, • ■ • , F m € S n be given. Inequalities consisting of any combination of the follow- 
ing relations 

m m 

F(x)=F + Y,XiFi>0, or F(x) = F + ]T Xl F t > 0, (6.1) 

i=l i=l 

are called LMIs with respect to the variable x = [x\, • • ■ , x m ) T € W". When the LMI is satisfied by a vector 
x we say that the LMI is feasible and x is a feasible point. 

Next, let us state some facts about general semi- definite programming (SDP, for short) problems and 
their duals. 
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Definition 6.2 Let c = (a, ■ ■ • , c m ) T € R m and F ,F 1 , . . . ,F m e S n be given. The following optimization 
problem 



min c T x, 



(6.2) 



(6.3) 



subject to F(x) = F Q + XjPj > 0, 

i=l 

is called a semidefinitc programming. Moreover, the dual problem of the SDP (6.2) is defined as 

max -Tr(P Z), 
subject to Z G <S n , Tr(ZF l ) = c i , i = 1,2, ■ ■ ■ ,m, Z > 0. 

The following basic assumption is imposed throughout this section. 
Assumption 6.3 The controlled MF-FSDE system [A, A, C, C; B, B, D, D] is MF-L 2 -stabilizable. 

For notational convenience, we rewrite the AREs (5.8) as follows 

TZ(P,Q,Q,R,R)=0, H{P,W,Q,Q,R,R) = 0, (6.4) 

where 

TZ(P, Q, Q, R, R) = PA + A T P + C 7 PC - (PB + C T PD)(R + D T PDy 1 (B T P + D T PC) + Q, 
H(P, n, Q, Q, R, R) = U(A + A) + (A + A) T U + (C + C) T P(C + C) + Q + Q 

- [U(B + B) + (C + C) T P(D + D)][R + R+(D + D) T P(D + D)] _1 
■ [(B + BfU +(D + D) T P(C + C)] . 

Lemma 6.4 Let Qi,Qi,Q 2 , Qi & S n and R\, R\, R 2 , R 2 S S m he given satisfying 

Qi _ Q2, Qi _ Q2, Ri < P2, Ri < R2- 

Assume that there exists (Po,Hq) such that 

n{P Q ,Q 1 ,Q 1 ,R 1 ,R 1 )>0, TZ(Po,U o ,Q 1 ,Q 1 ,R 1 ,R 1 )>0. 
Then there exist (P^IIf) and (P 2 *, 112; ) satisfying 

(n(P*,Q i ,Q i ,R i ,R i ) = 0, n(P*,Il*,Q i ,Q i ,R i ,R i ) = 0, fort = 1,2, 

\Pi<p 2 and ui<tl* 2 . 

Proof. By the assumptions of this Lemma, (Pq,Uq) must also satisfy 

K{P ,Q 2 ,Q 2 ,R 2 ,R 2 ) >0, H(Po,H ,Q 2 ,Q 2 ,R 2 ,R 2 ) > 0. 

It then follows from Proposition A. 11 that there exist (PJ* , IT*) and (P 2 , II 2 ), which arc the maximal solutions 
of their respective AREs: 

K(P*,Q i ,Q i ,Ri,R i ) = 0, TZ(P*,U*,Q l ,Q l ,R l ,R i ) = 0, fori = 1,2. 

Furthermore, (Pj^n*) must satisfy 

n{P^,Q 2 ,Q 2 ,R 2 ,R 2 ) > 0, K(P?,IL$,Q 2 ,Q 2 ,R 2 ,R 2 ) > 0. 

Hence P* < P 2 * and n* < II 2 because (P 2 *, n 2 ) is the maximal solution to its AREs. □ 
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Consider the following SDP problem 



max Tr(P) + Tr(II), 



subject to 





' PA + A T P - 


f C T PC - 


- Q 


PB 


f C T PD ' 




B T P -+ 


D T PC 




i?4 


D T PD 



>o, 



< 


n(A + A) + (A + A) T Ii 

+(C + C) T P(C + C) + Q + Q 


Ti{B + B) 


+ (C + C) T P(D 


+ D) 


> o 




(B + B) T U + (D + D) T P(C + C) 


R + R + 


(D + D) T P(D + 


D) 





(6.5) 



Theorem 6.5 Let Q , Q £ S n , R, R € S m be given. The following are equivalent: 

(i) There exists (P , n ) such that TZ(P ,Q, Q, R,R)>0 and K(P , H ,Q, Q, R, R) > 0. 

(ii) There exists a solution to the AREs (6.4). 

Moreover, when (i) or (ii) holds, the AREs (6.4) has a maximal solution (P*, IT*) which is the unique optimal 
solution to the SDP problem (6.5). 

Proof. We only need to prove that (i) implies (ii). Let Po be given as in (i). For any e > and e > 0, we 
have 1Z(P ,Q + el, Q + el, R,R) > and TZ{P ,U ,Q + eI,Q + eI,R,R) > 0. Applying Proposition A. 11 and 
Lemma 6.4, we have that for any positive decreasing sequence ei — > and ei — > there exists a decreasing 
sequence of symmetric matrices 



Pe > - >Pe i >Pe i+1 > P , 



> n fi > n e - i+1 > n 



such that 1Z(P ei , Q + ej, Q + ej, R, R) = and K(P ei , , Q + ej, Q + e t I, R, R) = 0. Hence the limit 
P* = lim P e and II* = lim n ? . exist and satisfy 

K(P*,Q,Q,R,R) = 0, TZ(P*,Il*,Q,Q,R,R) = 0. 

In addition, (P*,II*) must be the maximal solution of the AREs due to the arbitrariness of (Po,Ho)- By 
Schur's lemma (Lemma A.l), (P*,II*) is an optimal solution to the problem (6.5) due to its maximality. To 
prove the uniqueness, let (P*,IL) be any optimal solution to (6.5). Then Tr(P* — P*) + Tr(IT — IL) = 
as both (P*,IT) and (P*,IL) are optimal to (6.5). However, P* - P* > and n* - n* > since (P*,U*) 
is the maximal solution of (6.5). This yields P* — P* = and II* — II* = 0. □ 

As an immediate consequence of Theorem 6.5, we have the following result for the standard case Q, Q > 
and R, R > 0. 

Corollary 6.6 If Q,Q > and R, R > 0, then the AREs (6.4) admits a maximal solution (P*,IT) with 
P* , IT > which is also the unique solution to the SDP (6.5). In addition, if Q, Q > and R,R>0, then 
the maximal solution (P*,IT) with P*,IT > and the feedback control 

u*(t) = -(R + D T P*D)- 1 (B T P* +D T P*C)(X*(t)-E[X*(t)}) 

-(R + R+ (P + P) T P*(P + P))~ 1 [(P + P) T IT + (D + D) T P*(C + C)]E[X*(t)} 

is stabilizing for the system (1.1). 

Proof. When Q,Q>0 and R,R>0, (P , II ) = (0, 0) satisfies the LMIs 

K(P,Q,Q,R,R) >0, K(P,II,Q,Q,R,R) > 0. (6.6) 

Hence by Theorems 6.5 the AREs (6.4) admits a maximal solution (P*,n*). Moreover, by the proof of 
Theorems 6.5, P* > P = and n* > n = 0. If in addition Q, Q > and R, R > 0, then (P , ft ) = (SI, 51) 
solves (6.6) for a sufficiently small 5, 8 > 0. Hence P* > Po = 51 > and U* >Ho — 51 > 0. Moreover, by 
virtue of Proposition A. 10, the corresponding feedback control is stabilizing since (6.6) is strictly feasible in 
this case. □ 
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6.2 Optimal feedback Control 

In this subsection, we show that the value function of Problem MF-LQ can be expressed in terms of the 
maximal solution to the AREs (6.4). Moreover, if there exists an optimal control of Problem MF-LQ then 
it is necessarily represented as a feedback via the maximal solution to the AREs. 

Theorem 6.7 Assume that Theorem 6.5-(i) holds. Then Problem (MF-LQ) is well-posed and the value 
function is given by V(x) = x T Tl*x, \fx G R n , where (P*,IT) is the maximal solution to the AREs (6.4). 

Proof. The well-posedness has been shown in Theorem 5.2, which also yields V(x) = x T H*x. 
Now, for any fixed e > 0, the LMIs 

TZ(P,Q + eI,Q + eI,R,R)>0, K{P, Il,Q + eI,Q + el, R,R)>0 (6.7) 

are strictly feasible. Hence by Proposition A.ll, there is a maximal solution, denoted by (P £ ,LT e ), to the 
corresponding AREs 

TZ(P, Q + eI,Q + el, R, R) = 0, K(P, U, Q + el, Q + el, R, R) = 0. 

In addition, by Proposition A. 10, the feedback control u e (t) = r e (A e (<)— E[X e (t)fj +T e E[X e (t)] is stabilizing, 
where 

T £ = -(R+ D T P c D)- 1 (B T P e + D T P e C), 

F e = -(R + R+ (D + D) T P e (D + D)y 1 [(B + B) T YL t + (D + D) T P e (C + C)] . 
It is easy to verify that P e ,IT e ,r e and f e satisfy the following equations 

(A + BT e ) T P e + P,(A + BT e ) + (C + DK C ) T P ( {C + DT e ) = -Q - el - T^RT,, 
[A + A + Bf e + BY e )U e + U e (A + A + Bt e + ST £ ) T (6-8) 
+ (C + C + DT e + DT e ) T P e (C + C + DT t + DT e ) =-Q-Q-2eI- TJ(R + R)T e . 
Applying Lemma A. 4 to M = P e , N = H e and substituting u e (t) into (A.l), we have 

{<(Q + eI)X e (s),X e (s)) + <(Q + eI)E[X e (s)],E[X e (s)}) 
+(Ru e (s),u £ (s)) + (RE[u e (s)],E[u e (s)])Xds 

{<(Q + eI)(X e (s) - E[X e (s)}), (X e (s) - E[X e (s)})) + ({Q + Q + 2eI)E[X e (s)],E[X e (s)}) 
+ (R(u e (s)-E[u e (s)]),(u e (s) -E[u e (*)])) + ((R + R)E[u e (a)],E[u e (s)])}ds 

{{{Q + el + rf RT e )(X t ( s ) - E[X e {s)]),(X e {s) - E[X e {s)})) 
+ ({Q + Q + 2eI + TJ(R + R)T e )E{X e (s)],E[X e (s)]}\ds 

J { (X e (s) - E[X t ( s )]f[(A + BT e ) T P e + P e (A + BT e ) + (C + DT e ) T P e (C + DT e )] (X e (s) - E[X e (s)]) 
+E[X e (s)f [(C + C + DT e + DT e ) T P e (C + C + DT e + DT t ) 
+(A + A + BT C + ST e )U e + U e (A + A + BT e + BT e ) T ]E[X e (s)]}ds 

= -E\(X e (t) -E[X t {t)]) T P,(X t {t) -E[X e (t)])] + x T U e x ~E[X,(t)] T U,E[X £ (t)}. 
Since ( Hm E[(X € (t) - E[X e (t)]) T P € (X e (t) - E[X e (t)])] = and ^ hm E[X e (t)} T U e E[X t {t)} = 0, we obtain 

x T n f x = e 



E 



E 



i-t 

= E 



E 



[{{Q + eI)X e {s),X e {s)) + {{Q + eI)E[X e {s)],E[X e {s))) 
+ (Ru e {s),u e (s)) + (RE[u e (s)],E[u e (s)])}ds > V(x). 
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On the other hand, since P* = lim P e and II* = lim HL (similar to the proof of Theorem 6.5), we have 

e-X) e->0 

V(x) < x T H*x. This completes the proof. □ 

Corollary 6.8 Assume that Theorem 6.5-(i) holds. If there exists an optimal control of Problem (MF-LQ), 
then it must be unique and represented by the state feedback control 

U*(t) = T*(X*(t)-E[X*(t)]) + t*E[X*(t)}, 

where (P*,IT) is the maximal solution to the AREs (6.4), and 



(R + D T P*Dy 1 {B T P* + D T P*C), 

\Tr>*/n i n^ - 1 17 D i d\Ttt* i I n i f)\T i 



E 



E 



-E 



(R + R + (D + D) T P*{D + D)) [(B + B) T IY + (D + D) T P*(C + C)] . 
Proof. Let (X* (-),u* (■)) be an optimal pair of the LQ problem. Then a completion of squares shows 

' (QX*(s),X*(s)) + (QE[X*(s)],E[X*(s)]) + (Ru*(s),u*(s)) + (RE[u*(s)],E[u*(s)})}ds 
(X*(s)-E[X*(s)}),(X*(s)-E[X*(s)])) + ((Q + Q)E[X*(s)},E[X*(s)]) 

jo " 

+(R(u*(s)-E[u*(s)}), (u*(s) -E[u*(«)])) + ((R + R)E[u*(s)],E[u*(s)])}ds 

\x*(t) -E[X*(t)]) T P*(X*(t) -E[X*(t)]j\ +x T Il*x -E[X*(t)} T lTE[X*(t)} 

J {u*(s)-E[u*(s)]-T*(X*(s)-E{X*(s)]) 1 {R + D T P*D)~ 1 

*{a) -E[u*(s)] -T*(X*(s) -E[X*(s)])\ds 

+eJ^ {e[u*(s)]-T*E[X*(s)] T (R + R+{D + D) T P*(D + D)y 1 E[u*(s)] - f *E[X*(s)]jds. 

As u*(-) is stabilizing, we have 

lim E\(X*(t) -E[X*(t)}) T P*(X*(t) -E[X*(t)])] = 0, lim E[X* (t)fll*E[X* (t)} = 0, 
which implies 

V{x)=J{x,u*{.)) 

^x T U*x + Ej |u*(s)-E[u*(s)]-r*(X*(s)-E[X*(s)])J (R + D T P* D)^ 1 

■ u*(s)-E[u*{s)]-r*(X*(s)-E[X*(s)])\ds 

f°° ( - iT _ _ _i r - ~i 

+EJ |E[u*(s)]-r*E[X*(s)]J (R + R+(D + D) T P*(D + D)) |E[u*(*)] - T*E[X*(s)]jds. 

By Theorem 6.7 we have V{x) = x T U*x. Hence, 

lu*(s)-E[u*(s)]-T*(X*(s)-E[X*(s)}) (R + D T P*Dy 1 
( (s) -E[u*(s)) -T*(X*(s) -E[X*(s)])\ds = 0, 



(6.9) 



EJ |e[u*(s)] -T*E[X*(s)] (R + R + (D + D) T P*(D + D)) 1 E[u*{s)} -T*E[X*(s)]}ds = 0. 

As R + D T P*D and i? + -R + (-D + D) T P*(D + D) are constant positive definite matrices, u*{t) has to be 
in a feedback form u*(t) = T*(X*(t) - E[X*(t)}) + t*E[X*(t)}. □ 
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7 Numerical Examples 



In this section, we report our numerical experiments based on the approach developed in the previous 
sections. Note that the numerical algorithm we have used for checking LMIs or solving SDP [33]. 
The system dynamics (1.1) in our experiments is specified by the following matrices 





-0.7 


0.2 


-0.9 


-0.2 


-0.7 




-0.75 


0.25 


-0.95 


-0.25 


-0.75 




1.0 


-0.6 


2.0 


-0.6 


-0.8 




1.05 


-0.65 


2.05 


-0.65 


-0.85 


A = 


0.8 


0.8 


-1.7 


-1.5 


1.1 




0.85 


0.85 


-1.75 


-1.55 


1.15 




0.7 


-0.2 


0.1 


-0.3 


-0.2 




0.75 


-0.25 


0.15 


-0.35 


-0.25 




0.6 


-1.0 


-1.3 


0.6 


-0.2 




0.65 


-1.05 


-1.35 


0.65 


-0.25 




1.4 


-0.7 " 








1.45 


-0.75 " 










0.3 


-1.7 








0.35 


-1.75 








B = 


0.1 


-1.7 








B = 


0.15 


-1.75 


1 








-0.1 


0.1 








-0.15 


0.15 










0.4 


-1.2 








0.45 


-1.25 










0.1 


0.1 


0.2 


-0.1 


0.4 " 




0.15 


0.15 


0.25 


-0.15 


0.45 




-0.1 


-0.3 


0.2 


-0.1 


-0.3 




-0.15 


-0.35 


0.25 


-0.15 


-0.35 


C = 


0.6 


0.4 


-0.3 


0.1 


-0.2 


, c = 


0.65 


0.45 


-0.35 


0.15 


-0.25 




-0.1 


0.2 


-0.2 


-0.1 


0.1 




-0.15 


0.25 


-0.25 


-0.15 


0.15 




-0.2 


0.2 


0.3 


0.2 


-0.3 




-0.25 


0.25 


0.35 


0.25 


-0.35 




0.7 


-0.3 " 








0.75 


-0.35 " 










0.2 


-0.8 








0.25 


-0.85 








D = 


0.1 


-0.8 








D = 


0.15 


-0.85 










-0.1 


0.5 








-0.15 


0.55 










0.2 


-0.6 








0.25 


-0.65 









7.1 Numerical test of MF-L 2 stabilizability 

Since we have shown that the controlled MF-FSDE system is MF-L 2 -stabilizable in Proposition A. 5 if 
and only if (A. 6) is feasible (with respect to the variables X, X, Y and Y), we should check the MF- 
L 2 stabilizability first by tackling inequalities. After running the calculation of SDP program via Matlab 
software, the obtained feasible matrices X, X, Y and Y satisfy Proposition A. 5: 



Y = 



26.1032 


0.6379 


-7.9410 


1.4143 


-7.4032 


0.6379 


17.0911 


-0.4114 


8.2578 


1.3415 


-7.9410 


-0.4114 


19.4946 


1.1492 


14.0620 


1.4143 


8.2578 


1.1492 


21.8509 


7.8151 


-7.4032 


1.3415 


14.0620 


7.8151 


40.5193 


0.0471 


-0.0617 


0.0114 


-0.2361 


-0.0333 " 


-0.0617 


-0.1398 


-0.1104 


0.2431 


0.3623 


0.0114 


-0.1104 


0.0283 


0.1159 


0.0443 


-0.2361 


0.2431 


0.1159 


0.4583 


0.0880 


-0.0333 


0.3623 


0.0443 


0.0880 


0.0952 


-12.1167 


-1.8513 


7.0876 


-11.3987 


-1.6418 


0.9756 


2.1581 


5.2614 


-16.0940 


-12.8827 
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and 



Y 



-0.3539 -0.0281 -0.0278 -0.2997 0.1924 
-0.0070 -0.0900 0.1334 -0.4658 0.1065 



which give rise to the stabilizing feedback control law u{t) = K(X(i) — K[X(t)]) + Kl 
feedback gain 



with the following 



and 



K = YX~ 



K = YX~ 



-0.3725 0.1843 0.3405 -0.5390 
0.1689 0.5864 0.6700 -0.8716 



4.0644 1.2449 
4.2782 0.3405 



-3.7655 1.9996 
0.6593 0.7858 



-0.1289 
-0.3709 

-1.3936 
0.2849 



7.2 Numerical solutions of SARE 

Now we tackle the SARE (6.4) for the following Q, Q, R and R via solving the SDP problem (6.5): 
Q = diag([0,l,l,0,l]) and Q = diag([0,0.5, 1,0,0.5]), 

and 



fl = diag([l,l]) 
We then gain the following solution (P, IT) 



P 



and R = diag([1.5, 1] 



and 



n = 



0.4151 


0.3890 


0.2068 


0.0162 


-0.4059 


0.3890 


2.7208 


1.9097 


-2.6074 


-0.7756 


0.2068 


1.9097 


1.8535 


-1.8330 


-0.8979 


0.0162 


-2.6074 


-1.8330 


4.2403 


-0.2665 


-0.4059 


-0.7756 


-0.8979 


-0.2665 


2.1537 


0.6147 


0.5721 


0.2644 


-0.1455 


-0.6138 


0.5721 


4.2579 


2.8706 


-4.4158 


-0.6536 


0.2644 


2.8706 


2.6758 


-2.6653 


-1.0890 


0.1455 


-4.4158 


-2.6653 


6.8158 


-1.0674 


0.6138 


-0.6536 


-1.0890 


-1.0674 


3.1641 
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A Appendix 

A.l Some useful lemmas 

The "well-known Schur lemma in [10] plays a key technical role. 

Lemma A.l (Schur's lemma) Let matrices M = M T ,N and R — R T > be given with appropriate 
dimensions. Then the following conditions are equivalent: 
(i) M - NR- 1 ^ > (rcsp. >)0. 
M N 

> (rcsp. >) 0. 



(ii) 
(hi) 



N T R 
R N T 
N M 



> (rcsp. >) 0. 



In the original Schur lemma, the matrix R is required to be nonsingular. When R is possibly singular, we 
have an extended Schur's lemma making use of some generalized inverse matrices. To make it more precise, 
for any matrix M, there exists a unique matrix M + , called the Moore-Penrose inverse [31], such that 

MM+M = M, M+MM+ = M+, (MM + ) T = MM + , (M + M) T = M+M. 

Lemma A. 2 For a symmetric matrix S, we have 

(i) S+ = {S+) T . 

(ii) S > if and only if S+ > 0. 
(hi) SS+ = S+S. 

Its proof can be found in [1]. 
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Lemma A. 3 (Extended Schur's lemma) Let matrices M = M T ,N and R = R T be given with appro- 
priate dimensions. Then the following conditions are equivalent: 

(i) M - NR+N T > 0, R > 0, and N(I - RR+) = 0. 



(ii) 



(iii) 



M N 

N T R 

R N T 

N M 



> 0. 



> 0. 



Its proof can be found in [5] . 

Lemma A. 4 Let a constant matrix M,N G S n be given. Then for any admissible pair (X(-),u(-)) of the 
system (1.1), we have 



E| J \ i ({X{s)-E[X{s)]) T (A T M + MA + C T MC)(X(s)-E[X(s)}) 

+2(u(s) - E[u(s)]) T (B T M + D T MC)(X{s) - E[X(s)]) 
+ (u{s) - E[u(s)}) T D T MD(u(s) - E[u(s)}) 

+ ((C + C)E[X(s)] + {D + D)E[u(s)]) T M((C + C)E[X(s)} + (D + D)E[u{s)]) 
+E[X{s)f({A + A) T N + N(A + A)E[X(s)}) + 2E[X{s)f{A + A) T N(B + B)E[u(s)] dsj 



(A.l) 



= E 



(X(t)-E[X(t)\) M(X(t)-E[x(t)}) +E[X(t)] T NE[X(t)}- x T Nx, \/t > 0. 



Proof. Applying Ito's formula to (X{t) - E[X(t)]) M(X(t) - E[X(t)]), integrating from to t, and 
taking expectations, we easily get the desired result. □ 

Proposition A. 5 The following assertions are equivalent: 

(i) The controlled MF-FSDE system [A, A, C, C; B, B, D, D] is MF-L 2 -stabilizable. 

(ii) There exist matrices K, K and symmetric matrices X, X such that 



(A + BK)X + X(A + BK) T + (C + DK)X(C + DK) T 
+ (C + C + DK + DK)X(C + C + DK + DK) T < 0, 
(A + A + BK + BK)t + X(A + A + BK + BK) T < 0, X > 0, X > 0. 

In this case the feedback u(t) = K(X{t) - E[X(t))) + KE[X(t)\ is stabilizing. 
(iii) There exist matrices K, K and symmetric matrices X, X such that 

(A + BK) T H + X(A + BK) + (C + DK) T X(C + DK) 
+ (C + C + DK + DK) T X(C + C + DK + DK) < 0, 
^ (A + A + BK + BK)X + X(A + A + BK+ BK) T < 0, X > 0, X > 0. 



(A.2) 



(A.3) 



In this case the feedback u(t) = K(X(t) - E[X(t)\) + KE[X(t)] is stabilizing. 
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(iv) There are matrices K, K such that for any matrices Y, Y there exist unique solution X, X to the following 
matrix equations 



(A + BK)X + X(A + BK) T + (C + DK)X(C + DK) T 

+(C + C + DK + DK)X(C +C+DK+ DK) T + Y = 0, 
(A + A + BK + BK)X + X(A + A + BK + BK) T + Y = 0, X > 



(A.4) 



> 0. 



Moreover, if Y, Y > (rcsp. Y, Y > 0) then X,X > (resp. X,X > 0). Furthermore, in this case the 
feedback u(t) = K(X(t) - E[X(t)}) + KE[X(t)\ is stabilizing. 

(v) There are matrices K, K such that for any matrices Y, Y there exist unique solution X, X to the following 
matrix equations 



(A + BK) T X + X(A + BK) + (C + DK) T X(C + DK) 
+(C + C + DK + DK) T X(C + C + DK + DK) + Y = 0, 



(A.5) 



(A + A + BK + BK)X + X(A + A + BK + BK) T + Y = 0, X > 0, X > 



Moreover, ifY,Y > (rcsp. Y,Y > 0) then X,X > (rcsp. X,X > 0). Furthermore, in this case the 
feedback u(t) = K(X(t) - E[X(t)}) + KE[X(t)} is stabilizing. 



(vi) There exist matrices Y, Y and symmetric matrices X, X such that 



\ 



IA 1 



BY + Y T B T 



+(C + C+{D + D)YX- 1 )X(C + C+(D + D)YX- 1 ) T 



XC T + Y T D T 



CX + DY 



<0, 



-X 

X > 0, X > 



(A.6) 



^ (A + A)X +(B + B)Y + X(A + A) 1 +Y 1 (B + B) 1 < 0, 
In this case the feedback u(t) = YX-\X(t) - E[X(t)]) + Y%~ x E[X{t)\ is stabilizing. 
Proof. For any n u x n matrices K, K, define an operator $, $ : S n —> S n by 

$(X, X) = (A + BK)X + X(A + BK) T + (C + DK)X(C + DK) T 

+ {C + C + DK + DK)X(C + C + DK + DK) T , 
$(X, X) = (A + A + BK + BK)X + X(A + A + BK + BK) T . 

If X(-) satisfies the equation (1.1) with the feedback control u(t) = K(X(t) - E[X(t)]) + KE[X(t)], then 
by Ito's formula X(t) = E[(X(t) - E[X(t)])(X(t) - E[X(t)]) T ] and X(t) = E[X (t)]E[X (t)] T satisfy the 
differential matrix systems 

^-X(t) = $(X(i),X(i)) and ^-X(t) = d>(X(t),X(t)). 

KJbV (Alt' 

Applying the general result given in the appendix of [19], we have the equivalence between the mean-square 
stabilizability and each of the assertions (ii)-(v). Furthermore, with Y = KX and Y = KX the condition 
(A. 3) is equivalent to 

AX + XA T + BY + Y T B T + (CX + DY)X~ 1 (CX + DY) 
+{C + C+{D + D)YX- 1 )X(C + C+(D + D)YX- 1 ) T < 0, 



(A + A)X + (B + B)Y + X(A + A) T + Y T (B + B) T < 0, 



> 0, X > 0. 



Applying Schur's lemma (Lemma A.l) we have the equivalence of the assertion (vi). □ 

Let p* denote the infimum value of the primal SDP (6.2) and d* the supremum value of its dual (6.3). 
Then we have the following results ([33, 1]). 
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Proposition A. 6 p* = d* if either of the following conditions holds: 

(i) The primal problem (6.2) is strictly feasible, i.e., there exists an x such that F(x) > 0. 

(ii) The dual problem (6.3) is strictly feasible, i.e., there exists a Z G S n with Z > and Tr(ZFi) = 
Cj, i = !,■■■ ,m. 

If both conditions (i) and (ii) hold, then the optimal sets of both the primal and the dual are nonempty. In 
this case, the following complementary slackness condition 



F(x)Z = 

is necessary and sufficient for achieving the optimal values for both problems. 
Now we turn to rewrite the AREs (5.8) as 

K(P) = 0, K(P,IL) = 0, 

where 

Tl{P) = PA + A T P + C T PC - {PB + C T PD)(R + D T PDy 1 (B T P + D T PC) + Q, 
K(P,n)=tt(A+A) + (A+A) T II+(C+C) T P(C+C) + Q + Q- [U(B + B) + (C+C) T P(D + D)] 
■[R+R+{D + D) T P{D + D)Y l [{B + B) T U+{D + D) T P{C+C)\. 
In this subsection, we pose an additional assumption that the interior of the set 

v = {{P, n) e s n x s n | n(P) > o, K(P, n) > 0} 

is nonempty, namely, there exists a (P ,Uo) 6 S n x S n such that H(Pq) > 0, and TZ(Po,U ) > 0. 
Consider the following SDP problem 

max Tr(P) + Tr(n), 

> 0, 



(A.7) 



(A. 



subject to 



PA + A T P + C T PC + Q 


PB 


+- C T PD 


B T P + D T PC 


fit 


D T PD 



n(A + A) + (A^ 

+{C + C) T P(C- 


-A) T u 
f C) + Q + 


Q 


n(p + b) 


+ (C + C) T P(D + D) 


(B + B) T U + (D ^ 


-D) T P(C- 


-C) 


R + R + 


{D + D) T P{D + D) 



> o, 



(A.9) 



P - Po > 0, 

n - n > o. 

The constraints of SDP (A.9) can be equivalently expressed as a single LMI 



p(pn) 



' L(P) 














L(P,U) 














P-Po 














n-n 



>o, 



(A.10) 



where 



L(P) 



PA + A T P 


f C T PC - 


-Q 


PB 


±C T PD 


B T P -i 


-D T PC 




P4 


D T PD 



L(P,H) = 


Tl(A + A) + (A^ 
+ (C + C) T P(C- 


-i) T n 

f C) + Q + 


Q 


U(B + B) + (C + C) T P(D + D) 






-D) T P(C- 


-C) 


R + R+(D + D) T P{D + D) 
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Proposition A. 7 The dual problem of SDP (A.9) can be formulated as follows 



max -Tr(Q5 + WP + QS + WU Q ) - Tr(RV + RV), 

AS + SA T + BU + U T B T + CSC T + DUC T + CU T D T + DVD T + (C + C)S(C + C) T 

+ (D + D)U(C + Cf + (C + C)U T {D + D) T + (D + D)V(D + D) T + W + I = 0, 
(A + A)S + S(A + A) T + (B + B)U + U T {B + B) T + W + I = 0, 



subject to < 



S U T 
U V 



>o, 



s u T 

U V 



> 0, W > 0, W > 0, 



(A.ll) 



where S, S,W,W € S n , V, V £ S m and U,U G 



Proof. The constraints of the general dual problem (6.3) can be formulated equivalently as the constraints 
of (A.ll). To this end, define the dual variable Z G S in+2m for (6.3) as 



Z = 



' s U T 

U T 


Y? 


Y T 


Y T 


Yi 


S U T 

u f 


Y T 


Y T 


Y 2 


Y 4 


W 


Y T 




Y 5 


Y 6 


w 



> 0. 



By the general duality relation Tr (ZFi) = Cj,i = 1, ■ • ■ , m (see (6.3)) it follows that for any (P, II) G S n xS n , 

Tr([P(P, II) - P(0, 0)]Z) = -Tr(P) - Tr(n), 

which is equivalent to 

Tr ([AS + SA T + BU + U T B T + CSC T + DUC T + CU T D T + DVD T + (C + C)S(C + C) T 
+ (D + D)U(C + C) T + {C + C)U T (D + D) T + (D + D)V(D + D) T + W + l]P 
+ [{A + A)S + S(A + A) T + (B + B)U + U T {B + B) T + W + l]nj = 0. 

This leads to 

AS + SA T + BU + U T B T + CSC T + DUC T + CU T D T + DVD T + (C + C)S{C + C) T 

+ (D + D)U{C + C) T + (C + C)U T (D + D) T + (D + D)V{D + Df + W + I = 0, 
(A + A)S + S(A + A) T + (B + B)U + U T (B + B) T + W + I = 0. 

On the other hand, the objective of the dual problem (6.3) can be formulated as 

-Tr(F(0)Z) = -Tr(QS + WP Q + QS + WII n ) - Tr(RV + RV). 

In particular, since the matrix variables Y\, Y%, Y3, Y4, Y§ and Yq do not play any role in the above formulation, 
they can be dropped. Hence, the condition Z > is equivalent to 

> 0, W> 0, W > 0. 

u v 1 U V 

This completes the proof. □ 
We now show that the MF-L 2 -stability can be regarded as a dual concept of SDP optimality. 

Proposition A. 8 The dual problem (A.ll) is strictly feasible if and only if the controlled MF-FSDE system 
[A, A, C, C; B, B, D, D] is MF-L 2 -stabilizablc. 
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" s 


U T ' 


>o, 


' s 


U T ' 










u 


V 




u 


V 



Proof. First, assume that the controlled MF-FSDE system [A, A, C, C; B, B, D, D] is MF-i 2 -stabilizable 
by some feedback u(t) = K(X(t) - E[X(t)}) + KE[X(t)\. Let W > and W > be fixed. Then it follows 
from the assertion (v) of Proposition A. 5 that there exists a unique (S, S) satisfying 

(A + BK)S + S(A + BK) T + (C + DK)S{C + DK) T 

+(C + C + DK + DK)S{C + C + DK + DK) T + W + I = 0, 
(A + A + BK + BK)S + S(A + A + BK + BK) T + W + 1 = 0, 5 > 0, 5 > 0. 

Set U = KS and U = KS. The above relation can then be rewritten as 

AS + SA + BU + U T B T + CSC T + DUC T + CU T D T + DUS- X U T D T + (C + C)S(C + C) T 

+{D + D)U{C + C) T + (C + C)U T (D + D) T + (D + D)US~ 1 U T (D + D) T + W + I = 0, 
(A + A)S + S(A + A) T + (B + B)U + U T (B + B) T + W + I = 0. 

Let e > and e > 0, define V = el + US" 1 ^, V = eI + US~ 1 U T , W = -eDD T - e{D + D)(D + D) T + W 
and W = W. Then V, V, W and W satisfy 

AS + SA + BU + U T B T + CSC T + DUC T + CU T D T + DVD T + (C + C)S(C + C) T 

+ (D + D)U(C + C) T + {C + C)U T {D + D) T + (D + D)V{D + D) T + W + I = 0, 
01 + A)S + S{A + A) T + (B + B)U + U T {B + B) T + W + I = 0. 

Moreover, by Schur's lemma (Lemma A.l) for e > and e > sufficiently small we must have 

S D U ^ >o, W>0, W>0. 

Therefore, the dual problem (A. 11) is strictly feasible. 

Conversely, assume that the dual problem is strictly feasible. Then there exist S > 0, S > 0, U, U, V and 
V such that 

AS + SA + BU + U T B T + CSC T + DUC T + CU T D T + DVD T + (C + C)S(C + C) T 

+(D + D)U{C + C) T + (C + C)U T (D + D) T + (D + D)V{D + D) T < 0, 
(A + A)S + S(A + A) T + {B + B)U + U T {B + B) T < 0. 

It follows that 

AS + SA + BU + U T B T + CSC T + DUC T + CU T D T + DUS- 1 U T D T + (C + C)S(C + C) T 

+ {D + D)U(C + C) T + (C + C)U T {D + D) T + (D + D)US- X U T {D + D) T < 0, 
(A + A)S + S(A + A) T + {B + B)U + U T {B + B) T < 0. 

Define K = US^ 1 and K = US^ 1 . The above inequality is equivalent to 

(A + BK)S + S(A + BK) T + (C + DK)S(C + DK) T 

+{C + C + DK + DK)S{C + C + DK + DK) T < 0, 
(A + A + BK + BK)S + S(A + A + BK + BK) T < 0, S > 0, S > 0. 

We conclude that the assertion (iii) of Proposition A. 5 is satisfied. Hence, the controlled MF-FSDE system 
[A, A, C, C; B, B, D, D] is MF-L 2 -stabilizable. □ 

The following result presents the existence of the solution of the AREs (A. 8) via the SDP (A. 9). 
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S U T 
U V 



>0, 



Proposition A. 9 The optimal set of SDP (A. 9) is nonempty and any optimal solution (P*, II*) must satisfy 
the ARE (A.8). 

Proof. Proposition A.8, along with Proposition A. 6, yields the non-emptiness of the optimal set. Next, 
appealing to the complementary slackness condition (A. 7) in Proposition A. 6, we conclude that any optimal 
solution (P*,IT*) must satisfy 



' HP) 














L(p,n) 














P-Po 














n-n 



s U T 
U V 






Y 3 T ' 


Yx 


S U T 
U V 




Y 5 T 


Y 2 


Y 4 


W 




Y 3 


Y 5 


Y 6 


w _ 



where S, S, U, U, V, V, W and W are the corresponding optimal dual variables. From the above we can 
deduce the following conditions 



(A T P* + P*A + C T P»C* + Q)5 + (P*P + C T P*D)U = 0, 

(A T P* + P*A + C T P*C + Q)U T + (P,B + C T P*D)V = 0. 
(P T P, + D T P*C)S + (R + D T P*D)U = 0, 
(P T P* + D T P*C)U T + {R + D T P*D)V = 0, 

(u4A+A)+{A+A) T n r +(c+c) T P4c+c)+Q+Q^s+(u(B+B)+(c+c) T P4D+D[ 

(ll*(A+A)+(A+A) T Il*+(C+C) T P*(C+C)+Q+Q 



(A.12) 

(A.13) 
(A.14) 
(A.15) 
))U = 0, (A.16) 



U 1 



(B + Bfll* + (D + D) T P*(C + C))S+(R + R+(D + D) t P*{D + D) J U = 0, 
(B + B) T U, + (D + D) T P* {C + C))u T +(r + R+(D + D) t P* (D + D))v = 



n{B+B) + (C+C) T P*(D+D))V = 0, (A.17) 

(A.18) 
(A.19) 

(P. - P )W = 0, (A.20) 

(II* - IL )W = 0. (A.21) 

Hence (A.14) implies that U = -(R+D T P, t D)- 1 (B T P*+D T P :t C)S. Putting this into equation (A.12) leads 
to TZ(P^)S = 0. A same manipulation of equations (A.13) and (A.15) yields 1Z(P*)U T — 0. Similarly, (A.18) 
implies that U = -(R + R+ (D + D) T P*{D + D))" 1 [{B + B) T Il* + {D + D) T P*(C + C)]S. Substituting this 
into equation (A.16) leads to 7£(P*,n*)S = 0. And a similar same manipulation of equations (A.17) and 
(A.19) yields K(P*, IL)£/ T = 0. Recall that the dual variables S,S,U,U,V,V,W,W satisfy the following 
constraint 



AS + SA + BU + U T B T + CSC T + DUC T + CU T D T + DVD T + (C + C)S(C + C) T 
+ (D + D)U{C + C) T + (C + C)U T (D + D) T + (D + D)V{D + D) T + W + I = 0. 

Multiplying both sides of the above by 7?.(P»,IT») we have 

ll(P*)[CSC T + DUC T + CU T D T + DVD T + (C + C)S(C + C) T + (D + D)U{C + C) T 
+(C + C)U T (D + D) T + (D + D)V(D + D) T + W + I}K(P«) = 0. 

It follows from W > that 

K(P*)[CSC T + DUC T + CU T D T + DUS + U T D T + (C + C)S{C + C) T + (D + D)U(C + C) T 
+(C + C)U T {D + D) T + (D + D)US + U T (D + P>) T ]ft(P,) < 0. 
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(A.22) 



(A.23) 



Since 



S U T 
U T 



> 0, 



s U T 
U T 



>o, 



it follows from extended Schur's lemma (Lemma A.3) that V > US+U T , U = USS+, V > US+U T and 
U = USS + . By virtue of Lemma A. 2 we deduce the following 



CSC T + DUC T + CU T D T + DUS + U T D T 

= CSS + SC T + DUS+SC T + CSS + U T D T + DUS+U T D T 

= (CS + DU)S + {SC T + U T D T ) > 0, 



(A.24) 



and 



(A.25) 



(A.26) 



(C + C)S(C + C) T + (D + D)U(C + Cf + (C + C)U T (D + D) T + (D + D)US+U T (D + D) T 
= (C + C)SS+S(C + Cf + (D + D)US+S(C + C) T + (C + C)SS + U T (D + D) T 

+ (D + D)US + U T (D + D) T 
= ({C + C)S + (D + D)U^j S+ (§(C + Cf + U T (D + Df) > 0. 

Then it follows from (A.23) that TZ(P^)TZ{P^) < 0, resulting in K{P*) = 0. 

Recall that the dual variables S, S, T, T, U, U, W, W satisfy the following constraint 

(A + A)S + S(A + Af + (B + B)U + U T (B + B) T + W + I = 0. 

Multiplying both sides of the above by 7£(P*,IL), we have 

1l(P*,IL*)[W + I]H(P*,IL*) = 0. 

Since W > 0, we have TZ(P^H^) =0. □ 

The following result indicates that any optimal solution of the primal SDP gives rise to an MF-L 2 
stabilizing control of the MF-LQ problem. The readers can refer to [8]. 

Proposition A. 10 Let (P» , II*) be an optimal solution to the primal SDP (A. 9). Then the feedback control 
u(t) = T*(X{t) - E[X(t)]) + is stabilizing for the system (1.1), where 

r„ = -(R + D T P :t D)- 1 {B T P^+D T P,C), 

f, = -(R + R+ (D + Df P t (D + D))' 1 ({B + BflJ, + (D + D) T P*(C + C)). 

Proof. Let S, S, U, U, V, V, W and W be the corresponding optimal dual variables satisfying (A.12)- 
(A.21). First, we are to show that S > and S > 0. Suppose that Sx = and 5a; = 0,.t€ E". As U and 
U satisfy 

U = -(R + D T P sr Df 1 {B T P, + D T P*C)S 

and 

U=-(R + R+(D + DfP* (D + D))' 1 [(B + B) T U, + (D + D) T P*(C + C)] 
(see (A. 14) and (A. 18)), we also have Ux = and Ux = 0. The dual constraint (A. 22) then implies 

x T [CSC T + DUC T + CU T D T + DUS+U T D T + (C + C)S{C + Cf + (D + D)U(C + Cf 
+(C + C)U T {D + Df + (D + D)US + U T (D + Df + W + i] x < 0. 
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The same manipulation as in the proof of Proposition A. 9 gives x = 0. As S > and S > 0, we conclude 
that S > and S > 0. Now, the equalities (A. 22) and (A. 26) give 

AS + SA + BU + U T B T + CSC T + DUC T + CU T D T + DUS- X U T D T + (C + C)S(C + C) T 

+(D + D)U(C + Cf + (C + C)U T (D + Df + (D + D)US- 1 U T {D + D) T < 0, 
(A + A)S + S(A + A) T + (B + B)U + U T {B + B) T < 0, S > 0, S > 0, 

which is equivalent to the mean-square stabilizability condition (iii) of Proposition A. 5 with K = T* and 

ir = f*. □ 

Proposition A. 11 There exists a unique optimal solution to the SDP (A.9), which is also the maximal 
solution to the AREs (A.8). 

Proof. Let (P»,n*) be an optimal solution to the SDP (A.9). Proposition A.9 shows that (P*,II*) solves 
the AREs (A.8). To show that it is indeed a maximal solution, define 

r* = -(R + D T P^D)- 1 (B T P^+D T P t C), 

f» = -(R + R+ (D + D) T P,(D + D)y 1 [(B + B) T U, + (D + D) T P*(C + C)]. 
A simple calculation yields 

(A + BT+fP,, +P*{A + BY*) + (C + DTfP^C + DT,) = -Q - T^RT*, 
(A + A + BY * + Bt + IT (A + A + Bt „ + Bt *) T 
+(C + C + Dt » + Df *) T P*(C + C + Df * + Df *) = -Q - Q - ff(ii + i?)f *. 

On the other hand, it follows from Proposition A. 10 that it*(t) = T*(X*(t) - E[X*(t)]) + TJL[X*(t)] is a 
stabilizing control. A proof similar to that of Theorem 6.7 yields that (P*,II*) is the upper bound of the 
set V, namely, (P*,n») is the maximal solution. Finally, the uniqueness of the solution to the SDP (A.9) 
follows from the maximality. □ 
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